Skip to main content

Last updated by Errin O'Connor, Founder & Chief AI Architect, EPC Group

Azure AI Search: vector + keyword + semantic + hybrid retrieval + semantic ranker + indexing pipelines. Reference RAG stack for Microsoft enterprises: Azure AI Search + Azure OpenAI Service. Native Copilot Studio integration + Azure AI Foundry integration + custom app support. Chunking strategies: fixed-size / sentence / paragraph / structural / semantic / hierarchical (500-1500 tokens + 100-200 overlap typical). Pricing: Microsoft Standard S1-S3 tiers plus embedding costs, sized after discovery. Fit vs SharePoint search: large + heterogeneous + custom + programmatic. EPC Group engagements: Discovery, Single-Index build, Multi-Index build, Optimization, and an operations retainer (monthly, scoped to the estate) — each a fixed fee quoted after a scoping call.

Key Facts

  • Azure AI Search: vector + keyword + semantic + hybrid retrieval
  • Reference RAG stack: Azure AI Search + Azure OpenAI Service
  • Native Copilot Studio + Azure AI Foundry integration
  • Chunking: fixed-size / sentence / paragraph / structural / semantic / hierarchical
  • Azure AI Search S1 list price: about $248.20 a month per search unit at East US (Azure Retail Prices API, 2026-11-02); the bill scales with replicas × partitions
  • EPC Group Single-Index RAG: 8-week fixed fee; Multi-Index Enterprise: fixed fee quoted after discovery

The commercial path for the stages above is Azure AI consulting: discovery and design (three weeks, fixed fee), then the single-index build (eight weeks, fixed fee) or the multi-index enterprise programme quoted after discovery.

How to test a RAG answer: retrieval, grounding and completeness are separate checks

A citation shows which chunk the model was given, not that the answer is right. Test three things separately, with a fixed question set and known source documents:

  • Retrieval: did the expected documents come back? With relevance labels, Microsoft Foundry's Document Retrieval evaluator computes NDCG, XDCG, fidelity and holes; without labels, its Retrieval evaluator scores how relevant the returned chunks are to the question.
  • Grounding: does the answer stay inside the retrieved context? The Groundedness evaluator scores that, the precision side.
  • Completeness: does the answer include what the expected answer contains? Response Completeness (preview) needs a ground-truth answer, the recall side.

Hybrid queries merge keyword and vector results with Reciprocal Rank Fusion before any semantic reranking, so tune top-k, chunk size and query type against the retrieval scores, not against how fluent the answer reads. Include questions the corpus cannot answer and questions from users who must not see a document: the right result there is a refusal and no restricted content. The RAG evaluation set (CSV) has example questions, expected source documents, expected facts and the pass rule for each check; its rows are synthetic.

Sources: Microsoft Learn, Retrieval-Augmented Generation (RAG) evaluators (ms.date 2026-06-02); Hybrid search scoring (RRF) (ms.date 2026-06-08); Service limits for tiers and SKUs (ms.date 2026-09-16); Azure Retail Prices API (read 2026-11-02).

Azure AI Search RAG FAQ

What is Azure AI Search?

Azure AI Search (formerly Azure Cognitive Search) is Microsoft's enterprise search service supporting vector + keyword + semantic + hybrid retrieval. Six capabilities: (1) Vector search — dense embedding-based semantic retrieval. (2) Keyword search — traditional BM25 lexical retrieval. (3) Hybrid retrieval — combined vector + keyword (RRF fusion). (4) Semantic ranker — LLM-based reranking of results. (5) Knowledge base connectors — SharePoint, OneDrive, Azure Blob, Cosmos DB, Files. (6) Indexing pipelines — chunking, embedding, enrichment (OCR, entity extraction, translation). Foundation of enterprise RAG applications. Integrates natively with Copilot Studio + Azure AI Foundry + custom apps.

What is enterprise RAG (Retrieval-Augmented Generation)?

RAG grounds LLM responses in retrieved documents rather than the LLM's training data. Pipeline: (1) Ingest documents into vector store (Azure AI Search). (2) User query → embedding → similarity search returns top-N chunks. (3) Query + retrieved chunks → LLM prompt. (4) LLM generates response citing retrieved sources. Enterprise benefits: (1) Access to private data. (2) Up-to-date information. (3) Source attribution + citations. (4) Reduced hallucination. (5) Access control enforcement at retrieval time. Azure AI Search + Azure OpenAI Service is the reference RAG stack for Microsoft ecosystem.

How does Azure AI Search integrate with Copilot Studio?

Copilot Studio + Azure AI Search integration: (1) Native connector — Copilot Studio has built-in Azure AI Search knowledge source. Point to index, agent grounds responses in index content. (2) Filter integration — pass user context to filter results (security trimming). (3) Semantic reranking — leverage AI Search semantic ranker for higher-quality answers. (4) Hybrid retrieval — vector + keyword combined for better precision. Use case: Copilot Studio agent grounded in Azure AI Search index of enterprise docs (SOPs, contracts, KB) rather than SharePoint alone. Better performance than SharePoint-only for large corpora + heterogeneous formats.

What are chunking strategies?

Six chunking approaches for RAG: (1) Fixed-size — split at 512-1024 token boundaries. Simple, fast, may split mid-thought. (2) Sentence — split at sentence boundaries within size limit. Better semantics. (3) Paragraph — respect paragraph boundaries. Good for prose. (4) Structural — use document structure (headings, sections). Best for structured docs. (5) Semantic — LLM-based semantic segmentation. Highest quality, higher cost. (6) Hierarchical — retrieve both small chunks (for precision) + document summaries (for context). Best for complex enterprise content. Chunk size sweet spot: 500-1500 tokens with 100-200 token overlap. EPC Group RAG engagements optimize chunking per corpus.

What are typical Azure AI Search costs?

Azure AI Search bills its dedicated tiers per search unit, and a service uses replicas × partitions search units. Microsoft list prices per search unit at East US, for 730 hours a month (Azure Retail Prices API, read 2026-11-02): (1) Free — no charge; 3 indexes and 50 MB of storage on shared hardware, for development. (2) Basic — about $73.00 a month. (3) Standard S1 — about $248.20; S2 — about $978.20; S3 — about $1,963.70. (4) Storage Optimized L1 — about $2,803.20; L2 — about $5,606.40. Regional prices differ. Semantic ranker and embeddings are billed separately: Azure OpenAI text-embedding-3-small is $0.02 and text-embedding-3-large is $0.13 per 1M tokens on a Global deployment, slightly more on a regional one. A Serverless Developer tier is in preview and Microsoft does not recommend it for production.

What use cases fit Azure AI Search vs SharePoint search?

Six triggers for Azure AI Search over SharePoint search: (1) Very large corpus (100K+ docs) — SharePoint search quality degrades. (2) Heterogeneous formats — PDF + Word + Excel + emails + code + images. (3) Non-M365 sources — external docs, DBs, APIs, custom apps. (4) Advanced semantic + vector — SharePoint has basic semantic but not on par. (5) Custom relevance tuning — full control over scoring + reranking. (6) High-volume programmatic use — millions of queries per day. Use SharePoint search when: content lives naturally in SharePoint, standard M365 use, small-to-mid corpus. Use Azure AI Search when: large + heterogeneous + custom + programmatic.

How does EPC Group build Azure AI Search RAG?

EPC Group Azure AI Search RAG portfolio: (1) RAG Discovery + Design (3 weeks) — corpus analysis + architecture + fixed-fee proposal. (2) Single-Index RAG Deployment (8 weeks) — indexing pipeline + Azure OpenAI integration + Copilot Studio or custom app integration + evaluation. (3) Multi-Index Enterprise RAG (12-20 weeks) — multiple indexes with access control + advanced chunking + hybrid retrieval + semantic ranker + monitoring. (4) RAG Fine-Tuning + Optimization (8-12 weeks) — chunk strategy tuning + embedding model selection + evaluation + performance optimization. (5) Ongoing RAG Engineering Retainer — monthly, scoped to the estate. Each is a fixed fee set at scoping. All led by senior AI engineering architect.

RAG Discovery + Design

3-week fixed fee: corpus analysis + architecture + fixed-fee proposal. Call (888) 381-9725.

By submitting this form, you agree to our Privacy Policy. We respect your privacy and will never share your information.

Business Hours

Monday-Friday, 8 AM - 7 PM CT

Quick Response Guarantee

We respond to all inquiries within one business day

AI assistant — not a human