Prompt Injection Detection

Security Objective

Prompt injection detection identifies likely injection attempts and their consequences while a system runs. It combines signals from several layers of a request: the content that entered context, the model's behavior, the plan an agent proposes, the tool calls it makes and the actions that were attempted, allowed or denied. The result feeds a response policy, an alert queue and later investigation.

Detection matters because prevention is incomplete. Some injected content will reach the model, and some model outputs will reflect it. Detection shortens the time until someone notices, helps contain affected workflows and supplies the evidence needed to understand what happened. It does not prevent effects on its own.

Ownership Boundaries

Detection is broader than input / content inspection, which evaluates content before or during context assembly. Detection also watches what happens after the model has read that content. Plan drift detection and tool-chain analysis are specialised detection techniques listed separately in the defense taxonomy; this page covers their place in the overall detection architecture without replacing them.

Threat Model

The threat is text that changes what the model treats as instructions: typed by a user in a direct prompt injection, or embedded in retrieved data, messages, files or tool results in an indirect prompt injection. In agentic systems the concern extends to the plan and actions the model proposes, to propagation between agents and to persistence in memory or stores.

Assume the attacker can observe some outputs, can iterate, and may know which detectors exist. Assume also that attempts can be spread across turns, sources and steps so that no single event looks unusual. Detection is designed for these conditions and still cannot be complete.

Detection Results Are Evidence

Trust Model

Detectors are system components, but the telemetry they read and the scores they produce remain variable. Only the response policy that consumes detections, and the enforcement controls that act independently of them, are system decisions.

Three rules follow. Telemetry fields that record system decisions must come from the enforcing component, not from model text. A detector's output never flows into authorization as permission. And missing telemetry is represented as unobserved, never as clean.

Signal Layers

Individual layers are weak alone. Correlating them within one request, and across requests from the same principal, source or workflow, produces much stronger evidence than any single signal.

Enforcement Outcomes as High-Value Signals

Deterministic controls produce some of the most reliable detection signals. When agent authorization denies a proposed action, when information flow control blocks data from reaching an external sink, or when a human reviewer rejects an approval, the system has observed a concrete mismatch between what the model proposed and what policy allows.

Task Alignment and Plan Deviation

Many injections succeed by redirecting the task. Detection can compare the proposed plan and tool calls with the user's stated objective, the workflow's expected steps and the tools normally used for that task. Deviation is a signal, not proof: users change their minds and legitimate tasks vary. Dedicated plan drift detection formalizes this comparison; here it is one input to correlation.

Keep the representation of the user's objective authoritative. It should be captured by the application from the user's own turn or workflow definition, not reconstructed from model output that may itself have been influenced.

A controlled exercise, the Plan Drift Detection lab, practices distinguishing legitimate adaptation from material task or side-effect drift using authoritative application-side objectives. It is a deterministic simulation, not a detector test.

Sequence and Tool-Chain Signals

Individually plausible steps can combine into harmful effects: read a sensitive record, summarize it, then send the summary somewhere new. Sequence signals look at the chain rather than the step, for example sensitive reads followed by outbound actions, or new destinations appearing after external content entered context. Specialised tool-chain analysis deepens this; detection uses it as one layer.

Correlation Across Turns, Agents and Sessions

Attempts may span multiple turns, agent handoffs or sessions, for example when content written to memory affects a later request. Correlation needs stable identifiers: request and conversation IDs, provenance references for content items, agent and workflow identifiers and the principal on whose behalf actions ran.

For agentic and multi-agent systems, record each hop with its inputs and outputs so a detection can be traced back to the content that first entered context.

Detector Types

Record which family and version produced each detection. Combining families improves coverage, but the combined result is still probabilistic.

Detector Integrity and Isolation

A model-based detector reads the same untrusted content as the system it protects and can be influenced by it. Constrain its output to a fixed schema, give it no tools and no authority, and run it in a context separate from the main task. Where possible, prefer detectors that read structured telemetry over detectors that read raw natural language.

When a detector is unavailable or times out, record the gap as unobserved. Whether the workflow continues, degrades to a restricted mode or stops is a risk-tier decision for the response policy.

Inline, Near-Real-Time and Retrospective Detection

Inline detection must be fast and conservative in scope; it complements, but does not replace, deterministic checks at the same boundary. Retrospective detection is essential for persistent injection, because new detectors can find content that was stored before they existed.

Response Policy

A response policy outside the model maps detections, confidence, workflow risk tier and affected assets to actions. Higher-authority workflows justify lower thresholds and stronger responses.

Alert Quality and Triage

Alerts that analysts cannot act on are ignored. Each alert should carry the correlated evidence: which signals fired, the content items and provenance involved, the model output and proposed actions, and what enforcement decided. Group related signals into one case rather than emitting one alert per event.

Define severity from impact potential and observed effect rather than detector confidence alone. A low-confidence signal attached to a denied external data transfer can deserve more attention than a high-confidence content match in a read-only workflow.

False Positives, False Negatives and Drift

False negatives are assumed; downstream enforcement exists for them. False positives cost analyst time and user trust and push teams toward suppressions. Measure both separately and review every suppression rule with an owner and expiry.

Workflows, tools, content mix, model versions and attacker techniques change. Version detectors and thresholds, re-run a labelled regression set on each change and watch detection rates per workflow for shifts that indicate drift or silent breakage.

Telemetry Design

Telemetry can contain sensitive content and hidden context. Apply access control, minimization and retention rules at least as strict as those for the underlying data, and keep system decision fields separate from model-generated text.

Privacy and Retention

Detection needs enough data to reconstruct events, but full prompt and output logging can expose personal data, secrets and hidden instructions. Prefer references, hashes and excerpts where they suffice, restrict access to full records, and set retention by purpose. Record who accessed detection evidence.

Worked Scenario

Synthetic example. An internal assistant helps an employee prepare a vendor comparison. It can read shared documents, search the web and send email on the user's behalf. A retrieved web page contains a synthetic test marker phrased as an instruction to email an internal pricing sheet to an external address.

In a benign variant, the user explicitly asks to email a public summary to a colleague. The plan matches the objective, the data is public, the flow is allowed and the send step goes through the user's own confirmation. No alert is raised.

What the Scenario Proves and Does Not Prove

It shows correlation turning several weak signals and one deterministic denial into an actionable alert, and it shows the harmful effect being prevented by flow control, not by detection. It does not show that detection would catch a paraphrased or multi-session variant, that the web page author was malicious, or that the benign variant was risk-free. Prevention is attributed to IFC; visibility and containment to detection and response policy.

Incident Response Relevance

A detection starts an investigation. Conclusions about compromise follow from evidence of effects, not from the alert itself.

Controlled Testing

Test detection end to end with synthetic markers and labelled scenarios that exercise each signal layer: content, plan deviation, tool sequences and enforcement outcomes. Run tests in isolated environments with test identities, non-production data and harmless sinks. Label each scenario with the expected signals and response. Do not build or publish reusable payload libraries; the goal is to measure detection and response behavior.

Metrics

Each measure keeps its own denominator. There is no composite score, and benign task utility is reported separately.

Common Mistakes

Implementation Checklist

Standards and Sources

On source use: OWASP LLM01:2026 is the current classification for prompt injection. NIST AI 100-2e2025 is used only for terminology; it is not a detection standard. Microsoft's guidance on indirect prompt injection is vendor guidance that recommends layering probabilistic and deterministic mitigations and names detection-oriented techniques such as prompt shields, plan drift detection and tool chain analysis alongside information flow control, least privilege and human-in-the-loop. Microsoft's Prompt Shields documentation describes one product's detection of user prompt and document attacks within its documented scope; it is cited as an example, not as a general model. No detection efficacy figures are claimed on this page.

Frequently Asked Questions