Input / Content Inspection

Security Objective

Input and content inspection analyzes user input and externally sourced content before or during context assembly. It looks for prompt injection and related risk signals, classifies what it finds, and hands the result to a disposition policy that decides whether content is annotated, routed, quarantined, minimized, blocked or sent for review.

Its value is early, content-level visibility. Inspection can stop obviously unsuitable content from entering a context window, can route suspicious content to a more restricted workflow, and can produce evidence for telemetry and incident response. It cannot guarantee that content is safe, because model interpretation of language remains probabilistic and attackers adapt. For why inspection is one layer rather than complete prevention, see Why Prompt Injection Is Not Just Input Validation.

Ownership Boundaries

Inspection answers a narrow question about content. It does not decide who may act, what data may flow where, or what a component is permitted to touch. Agent authorization, information flow control and least privilege remain deterministic controls outside the model. How untrusted content is marked for the model is covered in spotlighting / data marking.

Inspection Results Are Evidence

Four distinctions keep inspection results from being misused as security decisions.

Threat Model and Limits

The threat is content that attempts to change what the model treats as its instructions, either typed by a user (direct prompt injection) or embedded in data the application retrieves or receives (indirect prompt injection). Related risk signals include attempts to extract hidden context, to request out-of-scope actions or to smuggle content into later steps.

Limits are structural. Natural language has no fixed boundary between data and instruction; the same sentence may be benign in one task and hostile in another; inspectors see a representation, not the model's eventual interpretation; and adversaries test against deployed inspectors. Inspection therefore reduces exposure and improves visibility but is never a complete security boundary.

Trust Model

The inspection engine and its output are separate objects. A system component can run the analysis faithfully, yet the score or label it produces remains variable evidence about variable content. Only the disposition policy that consumes the signal, and the enforcement controls further downstream, are system decisions.

Three rules follow. Parsing and inspecting content never upgrade its trust. The inspection signal never flows into authorization or flow decisions as a permission. And a missing or failed signal is represented as uninspected, never as clean.

Inspection Points in the Content Lifecycle

Ingestion-time inspection is cheaper and lets stored findings follow content, but it can go stale as inspectors improve. Request-time inspection sees the actual context and task but adds latency. Many systems combine both and record which inspector version produced each finding.

User Input Versus External Content

Direct and indirect content need distinct evaluation contexts. A user turn is expected to contain requests; the question is whether it tries to override system-level policy. External content is expected to be data; any instruction-like text inside it is a signal, because data should not direct the system at all.

Raw, Parsed, Normalized and Model-Visible Representations

Inspect the model-visible representation where possible, because that is what the model interprets. Retain a link from each finding to the raw representation and to the representation actually inspected, so an investigator can reproduce the result and see what the model received.

Text, Structured Fields, Documents and Multimodal Content

Different content types need different representations before inspection. Structured tool responses contain free-text fields that can carry language even when the schema is valid. Documents yield text through parsers that may include or skip metadata, comments or hidden layers. Images and audio reach text models through OCR, captions or transcriptions, and only those derived representations can be inspected by a text inspector.

For document and multimodal content, record which extraction path produced the inspected text. A clean OCR result says nothing about content that the OCR step did not capture.

Encoding and Canonicalization as Preprocessing

Content may be encoded, escaped or formatted in ways that make the same meaning look different. Normalizing representations before inspection helps inspectors compare like with like. This page intentionally gives no catalogue of encodings or obfuscation methods.

Rule-Based and Classifier-Based Signals

Combining families improves coverage, but combination is still probabilistic. Record which family produced a finding so dispositions and investigations can weigh it appropriately.

Semantic and Contextual Signals

Useful signals often depend on context rather than wording: instruction-like text inside a data channel, content that addresses the assistant or its tools, requests unrelated to the user's task, or references to hidden context. The same text may be ordinary in a user turn and suspicious inside a retrieved document, which is why evaluation context matters.

Provenance-Aware Inspection

Inspection can consume data provenance as context: entry path, source type, tenant or connector can select which inspector runs and which thresholds apply. Inspection does not own provenance, never writes findings into provenance fields, and never derives provenance from what content claims about itself.

Risk-Tiered Dispositions

A disposition policy outside the model maps findings, entry path and workflow risk tier to an action. Higher-authority workflows justify stricter dispositions and lower thresholds.

Inspector Trust and Service Failure

The inspector is a system component that processes untrusted content, and a model-based inspector can itself be influenced by that content. Keep its output constrained to a fixed result schema, never let it call tools, and treat its output as a signal only.

When the inspector is unavailable, times out or returns an unparseable result, the content is uninspected. What happens next is a risk-tier decision: high-impact workflows may block or route to a restricted path, while low-impact read-only workflows may proceed with the gap recorded. No single fail-open or fail-closed rule fits every application.

False Positives, False Negatives and Drift

False negatives let risky content through; downstream enforcement must assume they happen. False positives block or degrade legitimate work, which pushes teams to raise thresholds or add bypasses. Both must be measured separately rather than traded off silently.

Content mix, user behavior, inspector versions and attacker techniques change over time. Maintain a versioned, labelled regression corpus, re-run it on every inspector or threshold change, and watch finding rates for shifts that indicate drift.

Derived Content Re-Entering Context

Summaries, translations, tool results, agent messages and memory entries are new content produced from earlier content. When they re-enter a context window they need their own inspection, because a transformation can preserve or reshape instruction-like text. A clean finding on the source does not carry over automatically to the derivative.

Relationship to Downstream Controls

Worked Scenario

Synthetic example. A support assistant receives a user request to summarize an open ticket and propose a reply. Retrieval adds a customer-uploaded attachment whose extracted text contains a synthetic test marker phrased as an instruction to forward internal notes to an external address. The assistant has a send-email tool.

In a benign variant, an attachment containing ordinary troubleshooting steps raises no signal and is placed into context with its provenance. The summary is produced normally, and the send tool still requires the user's own confirmation.

What the Scenario Proves and Does Not Prove

It shows inspection producing an early signal that a disposition policy used to reduce exposure, and it shows authorization and flow control independently preventing the harmful effect. It does not show that inspection alone contained the attempt, that a paraphrased version would be caught, or that the benign attachment was trustworthy. Containment is attributed to the combination of gates.

Telemetry and Evidence

Findings can contain sensitive content. Apply the same access control and retention to findings as to the content they describe.

Incident Response for Suspicious Content

A finding is a starting point for investigation, not a conclusion that compromise occurred.

Controlled Testing

Test with synthetic markers and bounded, labelled corpora of benign and adversarial-style cases. Keep test content out of production stores, version the corpus, and label each case with entry path, modality and expected disposition. Do not build or publish a reusable payload library; the goal is to measure inspector behavior, not to maintain attack material.

Metrics

Each measure keeps its own denominator. There is no composite score, and benign task utility is reported separately.

Common Mistakes

Implementation Checklist

Standards and Sources

On source use: OWASP LLM01:2026 is the current classification for prompt injection. NIST AI 100-2e2025 is used only for terminology about adversarial machine learning and prompt injection; it is not a content-inspection standard. Microsoft's guidance on indirect prompt injection is vendor guidance that recommends a defense-in-depth strategy layering probabilistic and deterministic mitigations, listing prompt shields, spotlighting, plan drift detection, tool chain analysis, information flow control, least privilege and human-in-the-loop. Microsoft's Prompt Shields documentation describes one product that analyzes user prompt attacks and document attacks; it is cited as an example of product behavior within its documented scope, not as a general model.

Frequently Asked Questions