- Untrusted Input / External Content
- LLM / Agent
- Privileged Tool
The model can turn untrusted language directly into privileged action.
Prompt Injection Defense
Prompt injection defense requires layered controls because model interpretation remains probabilistic. Engineering must both reduce successful manipulation and limit security impact when manipulation occurs.
Security architecture
Defense-in-depth architecture
Separate model interpretation from permission to act. Preserve source context, validate proposed plans, and enforce consequential decisions at application boundaries.
- Untrusted Content
- Provenance / Inspection
- LLM
- Plan Validation
- Policy Engine
- Authorization
- Optional Approval
- Restricted Tool
- Monitoring
Independent controls preserve context, validate intent, authorize effects, and constrain execution.
- 01Context and provenance
Source, principal, tenant, classification, and trust status remain structured application data.
- 02Inspection and model robustness
Content and behavior provide risk signals without becoming the sole trust decision.
- 03Plan and policy
A proposed plan is checked against user intent, permitted workflow, and information-flow rules.
- 04Authorization
Principal, resource, action, parameters, and destination are authorized outside the model.
- 05Restricted execution
Restricted tools, sandboxing, egress rules, and specific approvals cap impact.
- 06Monitoring and recovery
Telemetry connects source, context, plan, action, and side effect for detection and incident response.
Control families
Defense domains
Resilience comes from complementary control families with distinct responsibilities.
DomainContext and provenance
Keep the origin and trust status of untrusted data outside model-generated text.
DomainModel-facing robustness
Design prompts, data marking, and model behavior to reduce the influence of competing instructions.
DomainDetection and monitoring
Correlate content risk, plan drift, tool chains, and anomalous side effects.
DomainIdentity and authorization
Bind authority to the initiating principal and check every consequential action.
DomainTool and action controls
Constrain capabilities, parameters, resources, and transactions explicitly.
DomainInformation flow and egress
Enforce permitted paths from source data to outputs, tools, and external destinations.
DomainContainment and recovery
Isolate execution, preserve evidence, rotate credentials, and restore safely.
DomainHuman oversight
Require informed and specific approval for selected risks.
DomainTesting and assurance
Exercise controls with representative attack paths and retest after change.
Enforcement
Probabilistic signals and deterministic enforcement
Detection signals help prioritize, block, or investigate suspicious behavior. They should not be the only protection around consequential operations.
| Control property | Probabilistic signals | Deterministic enforcement |
|---|---|---|
| Typical examples | Prompt injection classifier; content inspection; model-based critic; plan-drift signal | Authentication; resource and action authorization; parameter validation; destination allowlist |
| Decision character | Uncertain score with thresholds, false positives, and missed cases | Explicit rule over identity, state, resource, action, or destination |
| Appropriate role | Detect, prioritize, trigger review, or reduce risk | Enforce tenant, permission, sandbox, network, and side-effect boundaries |
| Security assumption | Some malicious content or behavior will remain undetected | Disallowed operations remain blocked even when model behavior is manipulated |
Application context
Defense by attack surface
The same principles apply across systems, but priority controls differ with the path into context and the authority available downstream.
Attack surfaceDirect Prompt Injection
Treat user input as untrusted, limit model authority, and authorize actions independently.
Attack surfaceIndirect Prompt Injection
Preserve external-content provenance, distrust tool output by default, and control information flow.
Attack surfaceRAG systems
Enforce retrieval permissions, tenant boundaries, document provenance, and permitted use of retrieved data.
Attack surfaceTool-using AI Agents
Control identity, plan validation, per-action authorization, parameters, egress, and side effects.
Attack surfaceMulti-agent systems
Retain message, delegated-authority, and provenance context across agent boundaries and reauthorize each handoff.
Operations
Operational security lifecycle
Defense is a maintained process, not a one-time prompt change.
- 01Threat Model
Map assets, principals, trust boundaries, content sources, tools, and possible side effects.
- 02Prevent
Reduce exposure, authority, and risky paths through architecture and least privilege.
- 03Detect
Correlate content, behavior, plan, tool, and action signals.
- 04Authorize / Contain
Block disallowed operations and tightly constrain permitted execution.
- 05Investigate
Reconstruct source, model context, plan, decisions, tool calls, and effects.
- 06Recover
Contain workflows, rotate credentials, restore integrity, and notify affected parties.
- 07Retest
Exercise the root cause and control gaps with representative scenarios.
Design rules
High-value design principles
These principles reduce the authority available to manipulated model behavior and improve investigation when controls fail.
PrincipleLeast privilege
Expose only the minimum tools, credentials, data, and actions needed for the current task.
PrincipleMinimize model-visible secrets
Keep secrets out of prompts when an external reference or short-lived capability will suffice.
PrinciplePreserve provenance outside the model
Maintain source and policy metadata structurally rather than expecting the model to reconstruct it from text.
PrincipleTreat external content as untrusted by default
Derive trust from source and use context, not from the tone or format of content.
PrincipleTrusted tool ≠ trusted tool output
A correctly implemented tool can return attacker-controlled websites, email, or records.
PrincipleModel output ≠ authorization
A model's tool call, plan, or explanation is a proposal, not a permission decision.
PrincipleValidate consequential actions
Check principal, resource, action, parameters, destination, and current state before execution.
PrincipleConstrain network egress
Permit only necessary destinations and protocols, and govern sensitive data flows separately.
PrincipleMake human approval specific
Show the concrete action, destination, data, and effect so consent is informed.
PrincipleBuild reconstructable telemetry
Trace source → context → plan → action → side effect with appropriate data minimization.
Control taxonomy
Defense knowledge areas
This taxonomy organizes the complete standalone defense library. Published controls link to their reviewed pages; Prompt Injection Prevention remains an architecture concept represented by the layered hub rather than a separate article.
Architecture and prevention
- Defense controlPrompt Injection Prevention
Layer controls that reduce manipulation opportunities and limit consequences when prevention is incomplete.
Context and provenance
- PublishedInput / Content Inspection
Inspect user input and external content for risk signals without treating inspection as a complete trust decision.
- PublishedData Provenance
Retain source, principal, tenant, classification, and retrieval context outside model-generated text.
- PublishedSpotlighting / Data Marking
Mark and transform untrusted data so its role is clearer to the model, while preserving external enforcement.
Detection and monitoring
- PublishedPrompt Injection Detection
Combine content, context, behavior, and action signals to identify suspicious influence attempts.
- PublishedPlan Drift Detection
Detect when an agent plan departs from the user's authorized objective or expected workflow.
- PublishedTool-Chain Analysis
Evaluate sequences of individually plausible tool calls for unsafe combined effects.
Identity, authorization, and oversight
- PublishedLeast Privilege
Limit identities, credentials, resources, tools, and actions to the minimum needed for the current task.
- PublishedAgent Authorization
Authorize the initiating principal, resource, action, parameters, and destination outside the model.
- PublishedHuman-in-the-Loop
Require informed, specific approval for selected high-impact actions rather than generic confirmation.
Information flow and action controls
- PublishedInformation Flow Control
Control whether data from a source may reach a model, action, tenant, or destination.
- PublishedNetwork / Egress Restrictions
Restrict outbound destinations, protocols, and data movement independently of model intent.
- PublishedOutput Validation
Validate structured outputs, parameters and destination formats against sink-specific structural and format constraints before use or execution.
Containment and recovery
Testing and assurance
Source orientation
Standards and guidance orientation
OWASP guidance and NIST terminology provide useful orientation, but neither should be treated as a complete defense standard or evidence that one control guarantees protection.
Reviewed orientation
- OWASP GenAI Security Project
- LLM01:2026 Prompt Injection
- NIST
- NIST AI 100-2e2025 terminology