What Is Multimodal Prompt Injection?
Multimodal prompt injection describes prompt-injection influence that is carried by inputs or representations which are not plain text alone: an image, a screenshot, a scanned page, an audio clip, a video frame, or a mixed artifact in which text and other modalities arrive together. The influence becomes effective only when a multimodal pipeline turns that artifact into something the model can read, interpret or attend to.
The entry mechanism is unchanged, and it applies only when the media actually carries prompt-injection content; an ordinary image upload is not, by itself, prompt injection. When injection-bearing media is supplied through the intended user input channel, the entry is direct prompt injection. When injection-bearing media reaches the model through externally retrieved or processed content, the entry is indirect prompt injection. What this article adds is the path in between: ingestion, rendering, extraction, transcription or visual interpretation, and the fusion of the result into one context.
Why Multimodal Is Not a Third Entry Mode
Filing multimodal cases next to direct and indirect suggests that a third channel exists. It does not. Every multimodal case is still one of the two entry modes, plus a representation property that changes what has to be inspected and what evidence can be collected.
| Aspect | Text-only case | Multimodal case |
|---|---|---|
| Entry | Direct or indirect | Direct or indirect, unchanged |
| Path to model visibility | The text is usually the representation | A decode, render, transcription or visual interpretation step sits in between |
| Inspectability | The model-visible string is normally recoverable | Some representations are recoverable, others are internal and not fully serializable |
| Human review | A reviewer reads the same text the model receives | What a reviewer notices and what the pipeline extracts can differ |
| Control surface | Input handling and context assembly | Ingestion policy, decoders, renderers, transcribers and context assembly as well |
Reading it as a property keeps the taxonomy stable. Nothing about trust boundaries, authorization or egress control changes because a pixel buffer was involved; what changes is the number of transformation steps between the source and the prompt.
Human-Salient, Machine-Extracted and Model-Visible
Precision of vocabulary is the most useful defensive habit in this topic. Three descriptions can all be true of the same artifact at the same time, and they are not interchangeable.
What a person reviewing the artifact is likely to notice under normal working conditions, which depends on the viewer, the display and the time spent.
What a decoder, renderer, OCR stage or transcriber actually produced from the artifact, which can be more, less or simply different.
What finally entered the model's context, either as extracted text or as an encoded representation of the media itself.
Pipeline Stages That Must Be Kept Apart
| State | What it establishes | What it does not establish |
|---|---|---|
| Artifact accepted or reachable | The media entered the ingestion path | That any modality was decoded |
| Modality decoded or extracted | A renderer, OCR, transcriber or encoder processed it | That the result reached model context |
| Representation model-visible | The derived content was placed into the prompt or encoded input | That behaviour changed |
| Measurable model influence | Output, plan or tool choice changed in a measurable way | That a policy was bypassed |
| Policy bypass | A control that should have stopped the step did not | That an action was executed |
| Unauthorized action | An action outside the authorized scope took effect | The extent of the impact |
| Security impact | A concrete effect on confidentiality, integrity or availability | Nothing further is assumed |
OCR Injection Is One Path, Not a Synonym
Many pipelines place a standalone text-recognition stage between an image and the model, and instruction-like text recovered there arrives as ordinary prompt text. That is one common path, and it is the easiest to evidence, because the extracted string can be logged and compared against the prompt.
It is not the whole topic. Vision-language systems can take an encoded representation of the image itself, without a separate recognition stage producing a string, and can respond to visual semantics rather than to recognised characters. In such systems there may be no extracted text to log at all, which changes both the detection strategy and what a report can honestly assert.
Modality Surfaces That Actually Reach the Model
A surface only matters when the deployed pipeline exposes it. The inventory below is a checklist to verify per system, not a list of properties every multimodal product has.
Instruction-like wording rendered into a photo, diagram, poster or chart that is uploaded or fetched.
Captures of chats, tickets, dashboards or terminals in which quoted content is indistinguishable in form from the surrounding interface.
Output of an OCR or text-recognition stage, which becomes ordinary prompt text. Where source, span or region-level provenance is available it should be preserved; otherwise the recognized text carries the source artifact's trust conservatively, and must not become more trusted because a system component produced it.
Encoded image input interpreted by a vision-language model without a separate recognition stage.
Audio processed by a transcription stage where the deployment uses one, including voice notes and call recordings.
Sampled frames or the separated audio track, where the pipeline processes video at all.
Document workflows in which the rendered page and the extracted text stream can differ; covered in depth under the document and PDF pillars.
Metadata, alt text, captions, annotations and embedded objects, which matter only where the pipeline actually reads them.
Scope and Neighbouring Classes
| Class | Defining mechanism | Relationship to this topic |
|---|---|---|
| Prompt Injection (LLM01:2026) | Inputs alter model behaviour | The parent class; OWASP lists a multimodal injection scenario under it |
| Adversarial examples | Perturbations shift a model's prediction or representation | A different objective in general, though research shows overlapping techniques in evaluated vision-language systems |
| OCR evasion | Text is shaped so recognition fails or misreads | Concerns extraction accuracy, not instruction influence |
| Document prompt injection | Document parts and parsing create model-visible content | Format-specific pillar; this page covers the modality property, not the file format |
| PDF prompt injection | Extraction, rendering, annotations and embedded files produce several representations | Format-specific pillar; render-versus-extract divergence is shared ground |
| Malware in media files | A crafted file exploits a parser or viewer | A classic software vulnerability class, not model influence |
| Training or model poisoning | Training data or weights are manipulated | A different lifecycle stage entirely |
Where a case is really about a file format, the document and PDF pillars carry the detail. Where it is about the fetch path, website prompt injection and tool output injection describe how external artifacts reach the system in the first place.
Authentic Origin Is Not Instruction Authority
A screenshot taken by a colleague, a signed PDF from a supplier and an authenticated upload from a logged-in customer are all authentic in the sense the security team usually means. None of that says anything about the instructions the content contains.
| Signal | What it establishes | What it does not establish |
|---|---|---|
| Valid signature | Integrity and authenticity of the content covered by that signature, within its validation scope | That the covered content may direct the system, or that every byte or update is necessarily covered |
| Authenticated uploader | Which account supplied the file | That the file content speaks with that account's authority |
| Internal source system | The artifact came from a known system | That the system was the author of every part of the content |
| Known file provenance | Where the artifact came from | Whether the content within it is task instruction or data |
The Multimodal Success Chain
Cross-Modal Fusion and Instruction Competition
Once several modalities are fused into one context, the model has to weigh the operator's task instructions against whatever arrived with the media. Nothing in the representation inherently marks which part is the task and which is the material to work on.
Research in this area is explicit about its scope. The USENIX Security 2025 paper on self-interpreting adversarial images introduces indirect, cross-modal injection attacks against visual language models in which images carry meta-instructions that steer how the model answers questions about the image, while the answers remain coherent and based on the visual content; the authors evaluate this across a set of models, interpretations and user prompts. Those findings belong to the systems and configurations the authors evaluated. They are not evidence that every multimodal model behaves this way, and this article reproduces none of the generation method.
Rendered Versus Extracted Representation
Where a pipeline can produce more than one representation of the same artifact, the difference between them is itself security-relevant. A rendered page, a recognised-text stream and a structural text extraction can each contain material the others do not.
| Representation | Typically produced by | Why it can differ |
|---|---|---|
| Rendered page image | A layout renderer | Reflects visual layering, overlays and what is actually drawn |
| Recognised text | An OCR stage over the rendering | Depends on contrast, size, language and recognition quality |
| Structural text extraction | A format parser | Follows the internal text objects, which need not match the visual result |
| Encoded media input | A vision or audio encoder | Not a string at all, so it cannot be diffed the same way |
Comparing representations is a useful detective control where two of them exist and are serializable. It is not available in every pipeline, and it is a signal for review rather than an authorization decision. Format-level detail for documents lives under PDF prompt injection.
Reachable Data and Capability Paths
Influence only becomes a security problem in proportion to what the influenced component can reach. The same manipulated screenshot in a read-only summarizer and in an assistant with payment and mail capabilities produces two completely different risk pictures.
What to enumerate for the influenced component
- Which data the session can read, and under whose identity
- Which tools and actions are callable in this workflow, not in the product overall
- Which outbound destinations are reachable, including link rendering and media fetch
- Which actions are consequential, meaning they change state, spend money or move data
- Which gates sit between a proposal and execution, and whether they are deterministic
- Whether influenced output is written to any state that a later task reads, which links to persistence
The agent-side consequences of an influenced plan are covered under AI agent prompt injection, and material that survives into later sessions under persistent prompt injection.
Offensive Assessment Perspective
In an authorized assessment, the value of this topic lies in mapping the media path stage by stage and recording where an attempt actually stops. The following are the assessment questions and observation points, not a construction procedure. This article deliberately contains no payload strings, no adversarial-image optimization methods, no concealment techniques and no exfiltration procedures.
Which media sources can an external party influence at all: uploads, linked images, shared documents, ticket attachments, tool results.
Which modalities the deployment actually ingests, and which are declined, stripped or ignored.
Which components run, in which order, with which limits, and what each of them emits.
What the prompt received, where that is serializable, and what evidence exists where it is not.
Whether media-derived content is presented as data with provenance, or merged into the instruction area of the prompt.
Which data and capabilities the influenced component could touch in this workflow.
Which authorization decisions are made outside the model, and which merely filter or score.
For each stage, what was recorded, so the report can state the deepest stage actually reached.
Worked Scenario: An Expense Assistant and a Screenshot
The following is a synthetic, bounded and reversible exercise in a test tenant with seeded data. It is described at the level of stages and decisions, without any reusable content.
A test expense assistant summarizes receipt images that employees upload and may call a reimbursement tool and a mail tool, both behind deterministic authorization.
A submitted receipt image contains additional instruction-like wording beside the receipt fields, phrased as if it were an operator note.
The image is accepted by the upload path. Evidence recorded: artifact hash, uploader identity, source channel, ingestion decision.
The recognition stage produces a text representation that includes the instruction-like wording. Evidence recorded: the extracted string and the transformation that produced it.
Context assembly places the extracted text into the prompt. In this configuration the representation is serializable, so the exact model-visible text is logged.
The generated plan changes: alongside the expected summary, the assistant proposes sending a message to an address that is not part of the workflow. This is measurable influence.
The mail tool call is evaluated by exact-action authorization against the workflow policy. Recipient and purpose are outside the authorized scope, so the call is denied before execution.
Deepest observed gate: exact-action authorization. Influence confirmed, no unauthorized action executed, no data reached an unauthorized destination, therefore no exfiltration. The test tenant is reset.
What the Scenario Does and Does Not Prove
| Claim | Supported? | Reason |
|---|---|---|
| The image path can carry influence | Yes | The extracted representation entered context and the plan changed |
| The model is broken | No | Observed influence depends on both context construction and model behavior, so it does not by itself establish a defect in one particular model |
| Every such image would work | No | One configuration, one workflow, one recognition stage were tested |
| Data was exfiltrated | No | The outbound call never executed |
| The deployment is safe | No | One gate held in one workflow; other workflows and other capabilities were not tested |
Detection and Telemetry
Reconstructing a multimodal case end to end requires visibility into each transformation, because the interesting question is which representation actually reached the model.
- 01Artifact identity
Hash, media type, size and the source channel or URL the artifact came from.
- 02Ingestion decision
Whether the artifact was accepted, declined, stripped or converted, and under which policy version.
- 03Transformations
Which renderer, recognition, transcription or encoding stage ran, with versions and settings.
- 04Derived representations
The extracted text or transcript where one exists, stored so it can be compared with the prompt.
- 05Context inclusion
Which derived content entered the context, in which segment, with which provenance labels.
- 06Plan and output change
The generated plan or output, so influence can be measured rather than inferred.
- 07Proposed tool call
The exact action requested, with parameters, before any authorization decision.
- 08Policy decision and outcome
Allow or deny, the rule that decided, and whether anything was executed.
Detection Limits in Multimodal Pipelines
Two limits deserve to be stated in every report rather than discovered during an incident.
First, not every model-visible representation is serializable. Where a vision or audio encoder feeds the model directly, there may be no text artifact that corresponds to what the model attended to. The artifact, the transformation record and the observed behaviour change remain available; the intermediate representation may not be.
Second, available evidence and inferred influence are different categories. That an image was ingested and a plan changed in the same turn is strong circumstantial evidence, not a causal proof. Controlled repetition with and without the artifact is what turns the inference into a measurement.
Media Filters and Classifiers Are Signals, Not Authorization
Modality-aware filtering has real value: recognition-based screening, instruction-likeness classifiers over extracted text, anomaly scoring on media sources and heuristics about content that does not belong in a receipt all reduce volume and support triage.
They are probabilistic. They produce false positives on legitimate artifacts such as screenshots of documentation, and false negatives on phrasing and layouts they were not trained for. They may run before a representation the model actually uses even exists. For all of those reasons they belong in the detection layer, and never in the position that decides whether an action may execute.
Defensive Architecture for Multimodal Input
- 01Media provenance
Record source, channel, uploader and fetch context for every artifact, and keep that record attached to everything derived from it.
- 02Trust labels through transforms
Preserve source, span or region-level provenance and trust through transformations where possible; a directly derived representation must not become more trusted because a system component produced it, and where only artifact-level trust is available, carry that source trust conservatively.
- 03Instruction and data separation
Present media-derived content as labelled data with provenance rather than merging it into the instruction area of the prompt.
- 04Allowlisted, sandboxed decoders
Accept known media types, run parsers and renderers with resource limits and isolation, and decline what the workflow does not need.
- 05Constrained media fetch
Restrict which remote media the system will retrieve, and treat every fetched artifact as external content.
- 06Representation comparison
Where two serializable representations exist, compare them and surface divergence for review as a detective signal.
- 07Least privilege
Scope the capabilities reachable in a media-handling workflow to what that workflow genuinely needs.
- 08Exact-action authorization
Decide each consequential action deterministically outside the model, bound to the concrete parameters at execution time.
- 09Egress and data-flow control
Constrain outbound destinations, including link and image rendering paths that can carry data outward.
- 10Human review for high impact
Require a person for irreversible or high-value actions in workflows that ingest externally controlled media.
- 11Plan-drift monitoring
Compare the executed plan against the requested task and flag steps that no part of the request accounts for.
- 12Transformation lineage
Keep the full chain from artifact to representation to context entry so an investigation can be reconstructed and derivatives located.
Risky and Stronger Pipelines Compared
- Any media accepted
- Decoded and merged into the prompt
- Model decides the next action
- Tool executes
Media-derived content arrives unlabelled in the same space as the task instructions, and the model's proposal is effectively the authorization decision.
- Allowlisted ingestion with provenance
- Decoded in isolation, trust label retained
- Labelled data segment in context
- Model proposes
- Deterministic exact-action authorization
- Egress control and logging
Every transformation is recorded, derived content preserves granular source trust and never becomes more trusted because a system component produced it, and a layer outside the model decides what may execute.
The stronger pipeline does not claim that influence is prevented. It claims that influence is visible, contained and unable to authorize itself, which is the achievable goal. The broader control set is described in the defense area.
Incident Response for Multimodal Artifacts
Response steps specific to media paths
- Preserve the original artifact, its hash and its source record before any cleanup
- Preserve every derived representation that exists, including extracted text and transcripts
- Record which transformation stages ran, with versions and settings
- Reconstruct the context entry: which segment the derived content occupied and with which labels
- Establish the deepest stage actually reached, from ingestion through to any executed action
- Determine whether any derived content was written to state a later task reads
- Classify the outcome precisely: extraction only, influence, contained proposal, unauthorized action or confirmed data exposure
- Retest the ingestion policy and the authorization gate before closing
Where derived content was stored, cleanup follows the lineage rules described under persistent prompt injection, because deleting the original image does not remove a summary that condensed it.
Controlled Testing
Setting up a defensible multimodal test
- Use a test tenant with seeded data and reversible state, never production records
- Fix the pipeline configuration under test and record decoder, recognition and model versions
- Run paired trials with and without the artifact so influence can be measured rather than inferred
- Instrument every stage separately so a stopped attempt can be attributed to a specific gate
- Test each modality the deployment actually ingests, and record which ones it declines
- Measure benign multimodal task utility in the same setup, so tightening ingestion is not mistaken for a security win
- Record coverage: which workflows, capabilities and modalities were in scope, and which were not
Metrics and How to Report Them
| Metric | Numerator | Denominator |
|---|---|---|
| Artifact Reachability Rate | Artifacts accepted into the ingestion path | Artifacts submitted or offered to it |
| Decode / Extraction Rate | Accepted artifacts whose relevant modality was decoded or extracted | Accepted artifacts |
| Model-Visibility Rate | Decoded representations that entered model context | Decoded representations |
| Cross-Modal Influence Rate | Model-visible cases with measurable behaviour change | Model-visible cases |
| Policy Bypass Rate | Influenced cases where a control that should have stopped the step did not | Influenced cases reaching that control |
| Unauthorized Action Rate | Actions outside authorized scope that took effect | Proposals reaching the authorization layer |
| Detection Recall | Media-path events detected by telemetry | Media-path events known to have occurred |
| False-Positive Rate | Benign artifacts flagged | Benign artifacts processed |
| Containment Rate | Cases stopped before an effect | Cases with confirmed influence |
| Representation Divergence Rate | Artifacts where two serializable representations differed materially | Artifacts where two serializable representations were deliberately compared |
Standards and Research Context
Reference frameworks
- OWASP
- LLM01:2026 Prompt Injection, including the multimodal injection scenario
- NIST
- AI 100-2e2025 terminology
- Microsoft
- Defend against indirect prompt injection attacks, Microsoft 365 Copilot prompt injection defense in depth
- Research
- USENIX Security 2025, self-interpreting adversarial images, 2026 arXiv work on image-based and cross-modal injection
OWASP LLM01:2026 states that the rise of multimodal AI introduces prompt injection risks arising from interactions between modalities, such as instructions placed in images that accompany benign text, that the complexity of these systems expands the attack surface, and that multimodal models may be susceptible to novel cross-modal attacks that current techniques find difficult to detect and mitigate. Its scenario list includes multimodal injection, in which a prompt embedded in an image accompanying benign text alters the model's behaviour when both are processed together. Robust multimodal-specific defenses are named there as an area for further research. None of that supports treating every multimodal case as indirect, nor assuming that an ingested image leads to impact.
NIST AI 100-2e2025 supplies the adversarial machine learning taxonomy and terminology used here. It is cited for vocabulary; this article does not claim that NIST standardizes Multimodal Prompt Injection as a separate class.
Microsoft's guidance on defending against indirect prompt injection and the Microsoft 365 Copilot defense-in-depth documentation are vendor guidance, identified as such. They are used here for the general architecture position: treat externally controlled content as untrusted, layer preventive, detective and limiting controls, and keep consequential decisions outside the content path.
On the research side, the USENIX Security 2025 paper Self-interpreting Adversarial Images by Zhang, Zhang, Morris, Bagdasarian and Shmatikov introduces indirect, cross-modal injection attacks against visual language models, using images that carry meta-instructions steering how the model answers questions about them while its answers remain coherent and grounded in the visual content. Two 2026 arXiv papers continue this line, one on image-based prompt injection against multimodal models through visually embedded instructions and one on cross-modal prompt injection against large vision-language models using image-only perturbation. All three are attributed research whose results belong to the authors' evaluated configurations, and no generation, optimization or concealment method from them is reproduced here.
Frequently Asked Questions
Is Multimodal Prompt Injection its own OWASP class?
No. It sits under LLM01:2026 Prompt Injection, which explicitly discusses multimodal risk and lists a multimodal injection scenario. The term describes a representation property, not a separate standardized class.
Is this the same as OCR injection?
No. Text recovered by a recognition stage is one path. Vision-language systems can interpret an encoded image directly without producing a recognised string, and audio paths behave differently again.
Does it only affect images?
No. Any modality the deployment ingests can carry it: screenshots, scanned pages, audio where a transcription stage is used, and video frames or audio tracks where video is processed.
Does the content have to be invisible to people?
No, and the claim should be avoided. Content may be visually subtle or simply overlooked, and what a pipeline extracts is a separate question from what a reviewer notices.
Is this the same as an adversarial example?
They are different classes with overlapping techniques in some evaluated systems. Adversarial examples target predictions or representations in general; prompt injection targets instruction-following behaviour.
Does a signed or internally sourced file change anything?
It authenticates the container, not the instructions. A successfully validated signature supports integrity and authenticity for the content covered by that signature within its validation scope; it says nothing about whether that content may direct the system, and it does not guarantee that every byte or update is covered unless that has been verified.
Is extraction already influence?
No. Recognised text in the logs proves extraction. Influence requires a measurable change in output, plan or tool choice, ideally shown by paired trials with and without the artifact.
Can media filtering solve this?
It reduces volume and supports triage. It is probabilistic, it misses and over-flags, and it must never be the component that decides whether an action may execute.
How is influence measured separately from extraction?
With separate rates and stated denominators: how often artifacts were accepted, how often the modality was decoded, how often the representation reached context, and how often behaviour actually changed.
What should be reported after a contained case?
Both facts: that a media-derived representation produced measurable influence, and that the deterministic gate denied the resulting action, naming the gate, the policy and the deepest stage reached.
References
- LLM01:2026 Prompt InjectionOWASP GenAI Security Project · GuidanceReviewed 2026-09-20
- Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025NIST · Standard · 2025-03Reviewed 2026-09-20
- Defend against indirect prompt injection attacksMicrosoft Learn · GuidanceReviewed 2026-09-20
- Microsoft 365 Copilot prompt injection defense in depthMicrosoft Learn · GuidanceReviewed 2026-09-20
- Self-interpreting Adversarial ImagesUSENIX Security 2025 · Research paper · 2025-08Reviewed 2026-09-20
- Image-based Prompt Injection: Hijacking Multimodal LLMs through Visually Embedded Adversarial InstructionsarXiv · Research paper · 2026-03Reviewed 2026-09-20
- A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only PerturbationarXiv · Research paper · 2026-05Reviewed 2026-09-20