Retrieval-Augmented Generation Platform on AWS

Overview

Retrieval-augmented generation (RAG) lets a large language model (LLM) answer questions using your own documents instead of relying only on what it learned during training. The model retrieves relevant passages from a search index and includes them in the prompt, which reduces hallucination and keeps answers current. Use this architecture when you need an LLM assistant grounded in private, frequently changing content such as policy manuals, support tickets, or product docs.

Components

• Amazon S3: stores raw source documents and the chunked, cleaned text used for indexing.
• AWS Step Functions + Lambda: an ingestion pipeline that extracts text, splits it into chunks, calls an embedding model, and writes vectors to the index.
• Amazon Bedrock: provides both the embedding model (to turn text into vectors) and the generation model (to write the final answer). Using a managed model service avoids running GPU servers.
• OpenSearch Serverless (vector engine): stores embeddings and performs approximate nearest-neighbor search to find the most relevant chunks.
• DynamoDB: holds conversation history, document metadata, and access-control tags.
• API Gateway + Lambda: the query service that orchestrates retrieval, prompt assembly, and the model call.

Data Flow

Ingestion runs asynchronously: a document lands in S3, an event triggers Step Functions, text is chunked and embedded, and vectors plus metadata are written to OpenSearch. At query time, the user question is embedded, the vector index returns the top-k chunks, the query Lambda builds a grounded prompt, and Bedrock generates an answer with citations back to the source documents. Responses and feedback are logged for evaluation.

Scaling and Resilience

All compute is serverless, so the system scales to zero when idle and absorbs spikes automatically. OpenSearch Serverless separates indexing and search capacity, letting each scale independently. Bedrock throughput is managed through provisioned or on-demand quotas; add a request queue with SQS to smooth bursts and apply backpressure. Deploy across multiple Availability Zones and keep the document store and index in the same region to limit retrieval latency.

Security

Ground every retrieval in the caller's permissions: store access tags on each chunk and filter the vector search by the user's entitlements so the model never sees documents the user cannot read. Validate and sanitize prompts to defend against prompt injection embedded in retrieved content. Encrypt data at rest with KMS and in transit with TLS. Apply least-privilege IAM roles to each Lambda. Log all prompts and completions for audit, and redact personal data before it reaches the model.

Trade-offs and Alternatives

Managed Bedrock models simplify operations but cost more per token than self-hosted open models on EC2 or SageMaker; choose self-hosting only when volume justifies the operational burden. OpenSearch Serverless is convenient but pgvector on Aurora can be cheaper at small scale and keeps vectors next to relational data. RAG is preferable to fine-tuning when knowledge changes often; fine-tuning suits stable style or format requirements. For very large corpora, consider hybrid search (keyword plus vector) and a re-ranking model to improve precision.