Retrieval-Augmented Generation (RAG)

Retrieval-augmented generation (RAG) combines a retrieval system with a language model. Rather than relying only on knowledge baked into the model's weights, RAG fetches relevant passages from an external source at query time and includes them in the prompt, so the model answers using current, specific, and verifiable information.

How It Works

A RAG pipeline first indexes a corpus by splitting documents into chunks and storing their embeddings. When a user asks a question, the query is embedded and used in vector search to find the most relevant chunks. Those chunks are inserted into the model's context window alongside the question, often with instructions to answer only from the provided material and to cite sources. The model then generates a response grounded in the retrieved text. Many systems add reranking or hybrid keyword search to improve relevance.

Why It Matters

RAG addresses key LLM limitations: it supplies up-to-date and domain-specific knowledge without retraining, reduces hallucination by grounding answers in real sources, and enables citations. It is often cheaper and faster to maintain than fine-tuning when knowledge changes frequently. Retrieval quality is the main lever on output quality, so chunking, embeddings, and reranking matter greatly.

Related Terms

RAG depends on embedding and vector-search for retrieval, fits results into the context-window, and helps an large-language-model avoid hallucination.