A Real RAG Pipeline on Azure: Internal Docs Q&A with Terraform (Model-Agnostic) 🔍

The same internal-docs Q&A problem every company has, solved on Azure AI Search and Azure OpenAI. Terraform provisions everything, model deployments are one-line swaps in tfvars, and a small evaluation catches regressions before they hit prod.

Same problem as every company: product docs, FAQs, and policy PDFs pile up in a shared drive, support agents can't find the right answer fast enough, and the same questions land in the same Slack channel every week. This post builds the fix - internal knowledge Q&A - on Azure AI Search paired with Azure OpenAI.

Azure's pattern connects three pieces in one call: Blob Storage holds your documents, Azure AI Search indexes them with vector and keyword search, and Azure OpenAI's chat completions API with a data source parameter handles retrieval, reranking, and generation together. Terraform provisions the infrastructure. Model deployments are variables, so upgrading either the embedding model or the generation model is a tfvars change, not a rewrite. 🎯

Docs (PDF/HTML/TXT) → Blob Storage

↓

A Real RAG Pipeline on Azure: Internal Docs Q&A with Terraform (Model-Agnostic) 🔍

Related reading

A Real RAG Pipeline on GCP: Internal Docs Q&A with Terraform (Model-Agnostic) 🔍

A Real RAG Pipeline on AWS: Internal Docs Q&A with Terraform (Model-Agnostic) 🔍

Zero-Trust RAG: Defeating the Shared Private Link Deadlock in Azure Terraform

RAG Powered Apps with Amazon Bedrock, Part 2: Automating the RAG Pipeline with…

Terraform for AI generated code

Building a RAG System From Scratch — Four Components, One Working Pipeline