LLM
139 items tagged with "llm"
Best Practices14
OpenAI Safety & Alignment Best Practices
Mitigation strategies (RLHF, red-teaming, tiered access) for large language model deployment.
Retrieval-Augmented Generation (RAG) Best Practices
RAG grounds a large language model in external documents retrieved at query time, reducing hallucination and letting answers reflect current, private data without retraining the model.
Prompt Engineering Best Practices
Prompt engineering is the practice of designing clear instructions, examples, and structure so a large language model returns accurate, consistent, and useful output.
LLM Evaluation and Evals
LLM evaluation measures model and application quality with repeatable tests, scoring accuracy, faithfulness, safety, and cost so teams can ship and improve with evidence.
LLM Guardrails
LLM guardrails are programmatic checks on model inputs and outputs that enforce safety, format, topic, and policy rules, blocking or correcting unsafe or off-policy responses.
AI Red Teaming
AI red teaming is structured adversarial testing of AI systems to find harmful, biased, or insecure behavior before attackers or real users do, using crafted attacks and probes.
Model Context Protocol (MCP)
The Model Context Protocol is an open standard that lets AI applications connect to external tools and data sources through a uniform client-server interface.
AI Agent Design Patterns
AI agent design patterns are reusable structures for LLM systems that plan, use tools, and act over multiple steps, covering reflection, tool use, planning, and multi-agent collaboration.
LLM Observability
LLM observability is the practice of tracing, logging, and measuring LLM applications in production to monitor quality, cost, latency, and safety and to debug failures.
OWASP Top 10 for LLM Applications (2025)
The OWASP Top 10 for LLM Applications lists the most critical security risks for generative AI systems, including prompt injection, sensitive data disclosure, and supply chain risk.
Prompt Injection Defense
Prompt injection defense protects LLM applications from attacks that hide malicious instructions in user input or retrieved content to override the system's intended behavior.
LLM Cost Optimization
LLM cost optimization reduces the spend of running language model applications through model selection, caching, prompt efficiency, and token-aware design without sacrificing quality.
Fine-Tuning vs RAG Decision Framework
A decision framework for choosing between fine-tuning, RAG, or both, based on whether the goal is new knowledge, consistent behavior, freshness, or domain adaptation.
Hallucination Mitigation
Hallucination mitigation reduces confident but false LLM output through grounding, retrieval, citation, verification, and uncertainty handling so answers can be trusted.
Tutorials7
How to build a RAG pipeline for question answering
Build a retrieval-augmented generation pipeline that grounds an LLM's answers in your own documents using chunking, embeddings, and a vector store.
How to build an LLM app with function calling (tools)
Give an LLM the ability to call your functions, so it can fetch data and take actions instead of only producing text.
How to fine-tune a language model on your own data
Adapt a base language model to your domain with supervised fine-tuning, covering data prep, training, and evaluation.
How to evaluate LLM outputs systematically
Build a repeatable evaluation suite for LLM features using reference checks, rubrics, and model-graded scoring.
How to engineer effective prompts for LLMs
Apply practical prompt-engineering techniques such as clear instructions, examples, and structured output to get reliable LLM results.
How to build an AI agent that plans and acts
Build an autonomous LLM agent that uses tools in a perceive-plan-act loop to accomplish multi-step tasks.
How to add guardrails to an LLM application
Protect an LLM app with input and output guardrails that filter unsafe content, block prompt injection, and validate structure.
Blueprints3
Self-Hosted LLM to Managed Inference Blueprint
Migrate self-hosted GPU LLM serving to a managed inference API to cut operational burden while controlling cost, latency, and data governance.
Keyword Search to RAG Platform Blueprint
Build a retrieval-augmented generation platform over your corpus with embeddings, a vector database, and grounded LLM answers replacing keyword search.
Ad-Hoc Prompt Scripts to AI Agent Platform Blueprint
Consolidate scattered LLM prompt scripts into a governed AI agent platform with tool use via MCP, guardrails, evals, and observability.
Reference Architectures9
Vector Search Platform for RAG
A vector search platform on Azure powering retrieval-augmented generation with embeddings, a vector database, and hybrid search.
Retrieval-Augmented Generation Platform on AWS
A reference design for a production RAG system on AWS that grounds LLM answers in private documents using a managed vector store and Bedrock models.
LLM Gateway and Proxy on Kubernetes
A reference design for a self-hosted LLM gateway on Kubernetes that centralizes routing, rate limiting, cost tracking, and guardrails across multiple model providers.
Multi-Agent LLM System on Azure
A reference design for a multi-agent application on Azure where specialized LLM agents coordinate through an orchestrator and shared tools to complete complex tasks.
LLM Fine-Tuning Pipeline on GCP
A reference design for fine-tuning open LLMs on GCP using Vertex AI custom training, parameter-efficient methods, and an evaluation gate before deployment.
LLM Observability Stack on Kubernetes
A reference design for an LLM observability platform on Kubernetes that captures traces, evaluations, cost, and quality signals across AI applications.
LLM Guardrails and Safety Layer on AWS
A reference design for an LLM safety layer on AWS that filters inputs and outputs, blocks prompt injection, and enforces content and PII policies.
Self-Hosted Open LLM Inference on Kubernetes
A reference design for serving open-weight LLMs on Kubernetes with GPU autoscaling, continuous batching, and an OpenAI-compatible API.
Conversational AI Assistant on Azure
A reference design for an enterprise conversational assistant on Azure that combines RAG, tool calling, and memory behind a governed orchestration layer.
Playbooks4
LLM and RAG Application Rollout Playbook
A program for rolling out a retrieval-augmented generation application from prototype to governed production with evals, guardrails, and observability.
AI Red-Teaming Program Playbook
A phased program to build an AI red-teaming capability that adversarially tests LLM systems for jailbreaks, prompt injection, and harmful outputs.
LLM Evaluation Program Playbook
A program to build a rigorous LLM evaluation capability with offline evals, online metrics, and regression gating in CI.
AI Agent Platform Program Playbook
A program to build a governed AI agent platform with tool integration via MCP, guardrails, evaluation, and observability.
Checklists5
LLM/RAG Production-Readiness Checklist
Verification items for taking a retrieval-augmented generation (RAG) application from prototype to reliable production service.
AI Red-Team Checklist
Adversarial test items for probing an LLM or AI application for prompt injection, jailbreaks, data leakage, and unsafe behavior.
LLM Evaluation Readiness Checklist
Verification items for building a trustworthy evaluation harness before releasing changes to an LLM-powered feature.
AI Agent Deployment Checklist
Pre-flight items for safely deploying an autonomous LLM agent that calls tools and takes actions on behalf of users.
LLM Cost Optimization Review Checklist
Review items for reducing the cost of an LLM application without degrading quality, covering prompts, caching, and model choice.
Regulations6
Executive Order 14110 on Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence
US federal executive order directing agencies to set safety, security, and rights standards for AI development and deployment across government and industry.
Colorado Artificial Intelligence Act
Colorado state law requiring developers and deployers of high-risk AI systems to prevent algorithmic discrimination in consequential decisions.
EU AI Liability Directive (Proposed)
Proposed EU directive easing the burden of proof for victims claiming damage caused by AI systems, complementing the EU AI Act.
Interim Measures for the Management of Generative Artificial Intelligence Services
Chinese regulation governing public-facing generative AI services, covering content safety, training data, and security review.
Artificial Intelligence and Data Act
Proposed Canadian federal law (part of Bill C-27) regulating high-impact AI systems for safety and non-discrimination.
ISO/IEC 42001 Artificial Intelligence Management System
International management-system standard for governing the responsible development and use of artificial intelligence within organizations.
Stacks8
RAG Stack (LangChain + pgvector + LLM)
Retrieval-augmented generation stack: LangChain orchestrates retrieval over pgvector embeddings in Postgres and grounds an LLM's answers in your own data.
LLM Agent Stack
Agentic AI stack: an orchestration framework drives an LLM to reason, call tools and APIs, and use vector memory to complete multi-step tasks autonomously.
vLLM + Ray Serve
A high-throughput LLM serving stack combining vLLM's optimized inference engine with Ray Serve for scalable, multi-replica deployment.
LangGraph + pgvector
An agentic LLM application stack using LangGraph for stateful agent workflows backed by Postgres with pgvector for retrieval and memory.
Haystack RAG
A production RAG stack built on the Haystack framework for composable retrieval-augmented generation pipelines over a document store.
LlamaIndex + Qdrant
A RAG and data-framework stack pairing LlamaIndex for LLM data ingestion and querying with Qdrant as a high-performance vector database.
Amazon Bedrock RAG
A managed RAG stack on AWS using Amazon Bedrock foundation models with Knowledge Bases for retrieval over data stored in S3.
MosaicML Composer Training
A large-scale model training stack using Composer and the MosaicML toolkit to train and fine-tune models efficiently on GPU clusters.
Comparisons10
RAG vs Fine-Tuning
Retrieval-augmented generation injects external knowledge at query time; fine-tuning bakes behavior into model weights. They solve different problems and often combine.
Fine-Tuning vs Prompt Engineering
Prompt engineering steers a model with instructions and examples in the context; fine-tuning changes the weights. Cost, control, and durability differ sharply.
GPT vs Claude
OpenAI's GPT and Anthropic's Claude are leading proprietary LLM families. They differ in design philosophy, context handling, and integration ecosystems rather than raw capability tier.
Open-Weight vs Proprietary LLMs
Open-weight models can be downloaded and self-hosted; proprietary models are accessed via API. The split shapes control, cost, privacy, and capability ceilings.
Llama vs Mistral
Llama (Meta) and Mistral are two leading open-weight LLM families. They differ in licensing, model sizes, mixture-of-experts use, and ecosystem maturity.
vLLM vs TGI
vLLM and Hugging Face Text Generation Inference (TGI) are high-throughput LLM serving engines. Both optimize GPU inference but differ in ecosystem and tuning focus.
LangChain vs LlamaIndex
LangChain is a broad framework for LLM application orchestration; LlamaIndex specializes in data ingestion and retrieval for RAG. They overlap but optimize for different centers of gravity.
Hugging Face vs OpenAI API
Hugging Face provides open models, tooling, and self-hosting paths; the OpenAI API offers managed access to proprietary models. The choice trades control against convenience.
Quantization vs Full Precision
Quantization stores model weights at lower bit-widths to cut memory and speed inference; full precision preserves maximum accuracy. The trade-off is size and speed versus fidelity.
AI Agents vs Workflows
Agentic systems let an LLM decide its own steps and tool use; workflows orchestrate LLMs through predefined paths. The choice trades flexibility against predictability.
Benchmarks39
MMLU (Massive Multitask Language Understanding)
A 57-subject multiple-choice benchmark testing broad academic and professional knowledge across STEM, humanities, social sciences, and law.
MMLU-Pro
A harder, reasoning-focused successor to MMLU with ten answer options and tougher questions designed to separate frontier models that saturated the original.
GSM8K (Grade School Math 8K)
A benchmark of ~8,500 grade-school math word problems that test multi-step arithmetic reasoning with a single numeric answer.
MATH (Competition Mathematics)
A benchmark of 12,500 competition-style math problems across algebra, geometry, number theory, and calculus, graded on exact final-answer match.
HumanEval
A code-generation benchmark of 164 Python programming problems graded by executing unit tests, popularizing the pass@k metric.
MBPP (Mostly Basic Python Problems)
A benchmark of ~1,000 entry-level Python programming tasks with test cases, used to evaluate basic code synthesis from short descriptions.
SWE-bench
A benchmark of real GitHub issues from open-source Python repositories where a model must produce a patch that resolves the issue and passes tests.
SWE-bench Verified
A 500-task, human-validated subset of SWE-bench with clear specifications and reliable tests, used as the standard clean measure of agentic coding.
BIG-bench (Beyond the Imitation Game)
A collaborative suite of 200+ diverse tasks probing reasoning, knowledge, and emergent abilities beyond conventional language benchmarks.
BBH (BIG-bench Hard)
A 23-task subset of BIG-bench focused on challenging multi-step reasoning where chain-of-thought prompting yields large gains.
HellaSwag
A commonsense sentence-completion benchmark where models pick the most plausible continuation among adversarially generated distractors.
ARC (AI2 Reasoning Challenge)
A grade-school science question benchmark split into Easy and Challenge sets, the latter built from questions retrieval methods answer incorrectly.
TruthfulQA
A benchmark measuring whether models avoid generating false answers that mimic common human misconceptions and falsehoods.
GPQA (Graduate-Level Google-Proof Q&A)
A benchmark of expert-written, graduate-level science questions designed to be extremely hard even with web access, testing deep domain reasoning.
MMMU (Massive Multi-discipline Multimodal Understanding)
A multimodal benchmark of college-level questions requiring joint reasoning over text and images such as diagrams, charts, and figures.
MT-Bench
A multi-turn conversational benchmark where a strong LLM judge scores model responses across categories on a 1-10 quality scale.
Chatbot Arena
A live, crowdsourced evaluation where users compare two anonymous model responses and votes are aggregated into Elo-style rankings.
HELM (Holistic Evaluation of Language Models)
A standardized framework evaluating language models across many scenarios and multiple metrics including accuracy, robustness, fairness, and efficiency.
DROP (Discrete Reasoning Over Paragraphs)
A reading-comprehension benchmark requiring discrete operations like addition, counting, sorting, and comparison over passage content.
WinoGrande
A large-scale commonsense benchmark of pronoun-resolution sentence pairs designed to require world knowledge rather than lexical cues.
AGIEval
A benchmark built from human standardized exams such as college entrance, law, and civil-service tests to measure human-centric reasoning.
LiveCodeBench
A contamination-resistant coding benchmark that continuously collects new competitive-programming problems and evaluates by execution over time.
AIME (Competition Math Benchmark)
An olympiad-level math benchmark using American Invitational Mathematics Examination problems with integer answers, a key frontier reasoning test.
tau-bench (Tool-Agent-User Benchmark)
An agentic benchmark testing tool-using models in simulated customer-service dialogues that require following domain policies and calling APIs correctly.
Terminal-Bench
An agentic benchmark evaluating models on completing real command-line tasks inside a sandboxed terminal, verified by automated checks.
RULER (Long-Context Benchmark)
A synthetic long-context benchmark with configurable tasks measuring a model's effective context length beyond simple retrieval.
IFEval (Instruction-Following Eval)
A benchmark that measures whether LLMs follow precise, verifiable formatting and content instructions using automatic checks rather than subjective judgment.
BFCL (Berkeley Function-Calling Leaderboard)
A benchmark for evaluating how accurately LLMs select, call, and parameterize functions and tools, including parallel, multiple, and multi-turn calling scenarios.
MGSM (Multilingual Grade School Math)
A multilingual extension of grade-school math word problems that tests whether LLMs can reason through arithmetic in many languages, not just English.
SimpleQA
A factuality benchmark of short, fact-seeking questions with single verifiable answers, designed to measure how often LLMs are correct, wrong, or appropriately abstain.
FRAMES (Factuality, Retrieval, And reasoning MEasurement Set)
A benchmark for retrieval-augmented generation that tests end-to-end factuality, multi-document retrieval, and multi-hop reasoning on questions needing several sources.
MuSR (Multistep Soft Reasoning)
A benchmark of long natural-language narratives requiring multistep commonsense and logical reasoning, such as murder mysteries and object-placement puzzles.
LiveBench
A contamination-resistant benchmark that continuously refreshes questions from recent sources and grades automatically against objective ground truth across many task categories.
BigCodeBench
A code-generation benchmark for realistic programming tasks that require composing many library calls, evaluated with rigorous test suites and high branch coverage.
CRUXEval (Code Reasoning, Understanding, and Execution)
A benchmark that tests whether models can reason about code execution by predicting function inputs from outputs and outputs from inputs.
RewardBench
A benchmark for reward models and LLM judges that measures how well they prefer better responses over worse ones across chat, reasoning, safety, and refusal cases.
AlpacaEval
An automated LLM-as-judge benchmark that estimates a model's win rate against a reference model on open-ended instructions, with a length-controlled variant to reduce verbosity bias.
Arena-Hard
An automatic benchmark of challenging, real-user-derived prompts graded by an LLM judge, built to align closely with human preference rankings and separate strong models.
HarmBench
A standardized red-teaming benchmark that measures how often automated attacks elicit harmful behaviors from LLMs and how well refusal and defenses hold up.
FAQs14
What is a large language model (LLM)?
A large language model is a neural network trained on vast amounts of text to predict the next token in a sequence, which lets it generate and underst...
What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation is a technique that supplements a language model with relevant documents fetched at query time, rather than relying onl...
What is the difference between fine-tuning and RAG?
Fine-tuning updates a model's weights by training it further on domain-specific examples, changing how the model behaves and what style or skills it h...
What is fine-tuning a model?
Fine-tuning is the process of continuing to train a pre-trained model on a smaller, task-specific dataset so it adapts to a particular domain, style, ...
What is a token in the context of LLMs?
A token is the basic unit of text that a language model reads and produces, typically a word fragment, whole word, or punctuation mark rather than a s...
What is a context window?
A context window is the maximum number of tokens a language model can consider at once, covering both the input prompt and the generated output. If a ...
What is prompt engineering?
Prompt engineering is the practice of designing the instructions, examples, and context given to a language model to get reliable, accurate outputs. T...
What is an AI agent?
An AI agent is a system that uses a language model to decide and take actions toward a goal, rather than producing a single response. It typically ope...
What is hallucination in LLMs?
Hallucination is when a language model produces text that sounds confident and plausible but is factually wrong or unsupported by its sources. It happ...
What is temperature in LLM generation?
Temperature is a parameter that controls the randomness of a language model's output by scaling the probability distribution over the next token. A lo...
What is a transformer architecture?
The transformer is a neural network architecture, introduced in 2017, that processes sequences using a self-attention mechanism instead of recurrence....
What is quantization in machine learning?
Quantization reduces the numerical precision of a model's weights and activations, for example from 32-bit floating point to 8-bit or 4-bit integers, ...
What is the Model Context Protocol (MCP)?
The Model Context Protocol is an open standard that defines how AI applications connect to external tools, data sources, and services in a consistent ...
What is chain-of-thought prompting?
Chain-of-thought prompting asks a language model to work through a problem step by step before giving a final answer, rather than responding immediate...
Glossaries20
Large Language Model (LLM)
A large language model is a neural network trained on vast text corpora to predict the next token, enabling it to generate and understand natural language across many tasks.
Token
A token is the basic unit of text an LLM processes, typically a word fragment, whole word, or character, produced by a tokenizer and mapped to a numeric ID.
Tokenization
Tokenization is the process of splitting raw text into tokens that a model can map to numeric IDs, usually using a subword algorithm such as byte-pair encoding.
Context Window
The context window is the maximum number of tokens a language model can consider at once, covering both the input prompt and the generated output.
Embedding
An embedding is a dense numeric vector that represents the meaning of text, an image, or other data so that similar items sit close together in vector space.
Vector Search
Vector search finds items whose embeddings are closest to a query embedding, enabling semantic retrieval by meaning rather than exact keyword match.
Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation is a technique that retrieves relevant documents at query time and supplies them to an LLM as context so its answers are grounded in external data.
Fine-Tuning
Fine-tuning is the process of further training a pretrained model on a smaller, task-specific dataset to specialize its behavior, style, or domain knowledge.
Prompt Engineering
Prompt engineering is the practice of designing and refining the text instructions given to a language model to steer its output toward accurate, useful results.
Inference
Inference is the process of running a trained model on new inputs to produce outputs, as opposed to the training phase that creates the model.
Training
Training is the process of adjusting a model's parameters from data so it learns to perform a task, typically by minimizing a loss function with gradient descent.
Transformer
A transformer is a neural network architecture built around self-attention that processes sequences in parallel, forming the basis of modern large language models.
Attention Mechanism
An attention mechanism lets a model weigh the relevance of different parts of its input when producing each output, focusing on the most pertinent tokens.
Hallucination
Hallucination is when a language model generates fluent, confident output that is factually incorrect, fabricated, or unsupported by its sources.
Temperature (LLM Sampling)
Temperature is a sampling parameter that scales an LLM's output probabilities, controlling how random or deterministic its token choices are.
Top-p Sampling (Nucleus Sampling)
Top-p sampling restricts an LLM's next-token choice to the smallest set of tokens whose cumulative probability exceeds a threshold p, then samples from that set.
Quantization
Quantization reduces the numeric precision of a model's weights and activations, shrinking memory use and speeding inference with limited accuracy loss.
AI Agent
An AI agent is a system that uses a language model to plan and take actions toward a goal, calling tools, observing results, and iterating with limited human input.
Model Context Protocol (MCP)
The Model Context Protocol is an open standard that defines how AI applications connect to external tools, data sources, and prompts through a uniform interface.
Foundation Model
A foundation model is a large model pretrained on broad data at scale that can be adapted, through fine-tuning or prompting, to a wide range of downstream tasks.