Prompt Injection Defense

Best Practice: Prompt Injection Defense

Prompt injection is an attack where malicious instructions hidden in user input or retrieved content trick an LLM into ignoring its original instructions. Direct injection comes from the user; indirect injection arrives through documents, web pages, or tool output the model reads. Defense is the layered practice of reducing the model's ability to be hijacked and limiting the damage if it is. It matters because, ranked as OWASP's top LLM risk, injection can lead to data theft, unauthorized actions, and policy bypass, and no single fix fully prevents it.

Researchers have shown repeatedly that no single technique fully prevents prompt injection, so defense is about layering controls and limiting blast radius. The most important architectural principle is to assume the model can be hijacked and to ensure that even if it is, it cannot do real harm. That means scoping tools to least privilege, requiring human approval before sensitive actions, isolating untrusted content from privileged contexts, and validating output before any downstream system acts on it. Indirect injection, where the payload hides in a document or web page the model reads, is especially dangerous in agentic and retrieval systems.

Step-by-Step Implementation Guidance

1. Treat all external content (user input, documents, tool output) as untrusted.
2. Separate trusted system instructions from untrusted data with clear delimiters.
3. Apply least privilege: scope tools so injection cannot trigger high-impact actions.
4. Require human approval for sensitive or irreversible operations.
5. Filter inputs and outputs with classifiers tuned for injection patterns.
6. Constrain output to validated schemas before any downstream use.
7. Isolate untrusted content processing from privileged contexts.
8. Red team continuously and add successful attacks as regression tests.

Concrete defenses stack into a defense-in-depth posture. Clearly delimit and label untrusted data so the model can distinguish instructions from content. Filter inputs and outputs with classifiers tuned for injection patterns. Constrain output to a validated schema before any system acts on it. For high-risk flows, a dual-LLM pattern, where a privileged model never sees raw untrusted text directly, limits exposure. None of these are sufficient alone, which is why continuous red teaming and least-privilege design remain essential.

Common Mistakes Teams Make When Ignoring This Practice

• Assuming a strong system prompt alone stops injection.
• Feeding retrieved web or document content straight into a privileged agent.
• Granting the model broad tool access without approval gates.
• Executing or rendering model output without validation.
• Never testing against known injection techniques.

Tools and Techniques That Support This Practice

• Defenses: input/output filtering, delimiter and structured prompting, dual-LLM patterns.
• Tools: Llama Guard, NeMo Guardrails, content classifiers.
• Process: least-privilege design, human-in-the-loop, AI red teaming.

How This Practice Applies to Different Migration Types

• Cloud Migration: Stop injected text in tickets from steering an ops agent into harmful commands.
• Database Migration: Prevent injected content from coercing unauthorized queries.
• SaaS Migration: Sanitize third-party API responses before an agent acts on them.
• Codebase Migration: Block comments or docs in legacy code from hijacking a coding agent.

Checklist

• All external content treated as untrusted
• Instructions separated from data
• Tools scoped to least privilege
• Human approval for sensitive actions
• Input and output filtering in place
• Output constrained to validated schemas
• Injection attacks covered by regression tests