What Is System Prompt Extraction?
System prompt extraction is the objective of making a model reveal, reproduce or reconstruct hidden context that the application placed into its context window but did not intend to show to the requester. The NIST CSRC glossary defines prompt extraction as an attack that tries to divulge the system prompt or other information in the context of a large language model that would normally be hidden from a user, following the terminology of NIST AI 100-2e2025.
The system prompt is the most visible member of that hidden context, but not the only one. Developer instructions, retrieved policy text, tool schemas, routing rules, configuration fragments and internal identifiers can all sit in model-visible context without being intended for users. Extraction targets any of them.
Objective vs. Mechanism
The site's attack taxonomy separates entry paths from objectives. Direct and indirect prompt injection are entry paths: they describe how attacker-controlled content reaches the model. System prompt extraction is an outcome class: it describes an intended result, regardless of which path carried the attempt.
| Question | Prompt injection | System prompt extraction |
|---|---|---|
| What it describes | How competing instructions enter model context | What the attacker wants disclosed |
| Typical entry | Direct user input or external content | Often a prompt-injection attempt, but also configuration or log exposure outside the model |
| Success condition | Model behaviour changes toward attacker intent | Hidden context is reflected or reconstructed in output |
| Severity driver | What the influenced model can reach or do | What the disclosed context actually contains |
Not every extraction attempt is a prompt injection in the strict sense, and not every prompt injection aims at extraction. An extraction attempt can succeed while a behavioural manipulation attempt fails, and the reverse. Reports should say which of the two was observed.
The Model-Visible Context Boundary
The decisive boundary is not whether content is labelled as a system prompt. It is whether the content is placed into the model's context at all. Everything the model can read can, in principle, influence or be reflected in its output. Material outside the current model-visible context cannot be extracted from that context through model output, because the model never saw it. This holds for the current context only: if a later authorized tool call or retrieval path returns that material, or information derived from it, into model-visible context, it becomes a separate tool, data-flow or retrieval exposure that must be controlled independently. Credentials and authorization decisions should therefore stay outside model-visible context permanently, not merely outside the current prompt.
Applications assemble context from many sources: developer instructions, user input, retrieval results, tool schemas and tool output, memory and conversation history. The assembly code decides what becomes model-visible. A reviewer asking about extraction exposure is therefore asking a concrete architectural question: what does this application put into the context window, and what does it deliberately keep outside?
What Hidden Context Typically Contains
Role, task, style and behavioural instructions the application prepends.
Criteria for refusals, escalations, tool selection or workflow branches.
Names, parameters and descriptions of tools the model may propose.
Policy documents, configuration snippets or profile data pulled into context by retrieval.
Tenant names, endpoint names, queue or table references, deployment details.
Credentials, connection strings or tokens - an architectural defect, not an extraction technique.
The same extraction output can be harmless in one deployment and serious in another. What matters is which of these constituents the application actually placed in context, not the fact that a system prompt exists.
Degrees of Disclosure
Extraction is not binary. A report that treats any output resembling an instruction as full compromise overstates the finding; a report that dismisses anything short of verbatim disclosure understates it. Four degrees should be distinguished.
| Degree | What happened | What it establishes |
|---|---|---|
| Exact disclosure | Output reproduces hidden context verbatim or near-verbatim | The content was model-visible and was reflected |
| Partial disclosure | Output reproduces identifiable fragments or sections | At least parts of the hidden context were model-visible |
| Semantic reconstruction | Output paraphrases rules, roles or criteria without quoting them | The model can describe its operating instructions; wording may be inferred |
| Adjacent context disclosure | Output reveals configuration, tool schemas or identifiers rather than instruction text | Other model-visible context was exposed, with its own impact assessment |
Prerequisites and Reachable Hidden Context
An extraction attempt can only disclose what is actually reachable. Three prerequisites must hold before any degree of disclosure is possible: hidden context must exist in the model-visible context of the target deployment; the attempt must reach a model that has that context assembled; and the model's output must reach the requester. If any of the three fails, there is nothing to extract through the model.
Reachability varies by deployment. Some applications place only a short behavioural instruction in context; others assemble large hidden contexts with policy text, schemas and configuration. Some deployments expose the same context to every requester; others assemble per-tenant or per-role contexts, which changes who can reach what. A review that does not establish what was assembled cannot establish what was exposed.
The Extraction Success Chain
Extraction is a chain of distinct conditions, each of which can hold or fail independently.
The chain framing matters because defensive work can break it at several independent points: minimizing what enters context, constraining what output may reflect, and ensuring that nothing security-critical depends on hidden context staying secret.
Distinct Security Facts
A request being accepted is not the same as hidden context existing, which is not the same as the context being reflected, which is not the same as exact extraction, which is not the same as sensitive disclosure, which is not the same as a control bypass, which is not the same as downstream impact. Each is a separate, checkable fact.
The application processed the interaction. Says nothing about exposure.
Established by inspecting the assembled context, not inferred from output alone.
Output shows overlap with hidden content; degree to be classified.
The disclosed material actually aids an attacker or violates a policy.
A separate chain: disclosure must enable something beyond the disclosure itself.
This separation keeps reports honest and keeps remediation proportionate. Exact disclosure of a non-secret routing instruction and disclosure of an embedded credential are both extractions, but they are not the same incident.
Offensive Perspective: Review Questions
For authorized review of one's own systems, extraction testing is about reachability and degree, not about collecting clever phrases. This article deliberately does not provide extraction prompts, payload catalogs or obfuscation recipes.
Questions an authorized review answers
- What does the application place in model-visible context, and is any of it sensitive by policy?
- Can output from the normal requester channel reflect that context, in full, in part or as a paraphrase?
- Does externally supplied content processed by the system (the indirect path) produce the same exposure?
- Do per-tenant or per-role contexts leak across boundaries when reflected in output?
- Would disclosure change any real security decision, or is hidden context already treated as non-secret?
- Do logs and telemetry record enough to reconstruct what was assembled and what was returned?
A review that answers these questions with evidence is more useful than a screenshot of a model reciting instructions, because it separates the reachability finding from the impact assessment.
Direct and Indirect Paths to the Same Objective
The objective can be pursued through both entry paths. Through the direct path, the requester's own input steers the model toward revealing its context. Through the indirect path, content the system retrieves or processes - a page, a document, a retrieval result or tool output - contains text aimed at the same disclosure, for example steering an agent to include its hidden instructions in an outbound summary or a shared artifact.
The indirect variant is easy to underestimate because the requester may be a legitimate user who never typed anything hostile: the disclosure is triggered by content and lands in output that user receives, or in a stored artifact that survives via persistent state. The classification of entry path and the assessment of exposure remain separate questions.
Neighbouring Classes and Distinctions
| Class | Defining mechanism | Relationship |
|---|---|---|
| Prompt injection | Competing instructions enter model context | A frequent mechanism for extraction attempts; not the objective itself |
| Jailbreaking | Bypassing safety restrictions on model behaviour | Different objective: prohibited behaviour, not disclosure of hidden context |
| Training-data extraction / model inversion | Recovering training examples or model internals | Targets the model's learned data, not the context assembled at runtime |
| Sensitive information disclosure | Output reveals protected user or business data | Overlaps when hidden context contains such data; otherwise a different impact class |
| Tool discovery | Learning which tools or schemas a system exposes | A subset of hidden-context exposure when schemas are model-visible |
| Model extraction | Replicating model weights or behaviour through queries | Targets the model artifact, not the application's hidden instructions |
The boundaries matter for classification. OWASP's 2026 edition places the runtime-context exposure under LLM08:2026 Hidden Context Exposure and the injection mechanism under LLM01:2026 Prompt Injection, and assigns regulated user or training data leakage to LLM02:2026 Sensitive Information Disclosure. One incident can touch several codes.
Why the System Prompt Must Not Be a Secret
The central architectural rule: anything placed in model-visible context should be treated as potentially discoverable. OWASP's 2026 guidance states the design assumption directly - practitioners should design under the assumption that hidden context is discoverable, should ensure that disclosure has little or no direct security impact, and should not embed credentials or rely on hidden context as a security boundary for authorization, privilege separation, policy enforcement or content filtering.
Microsoft's security planning guidance for LLM-based applications makes the same point from the vendor side, listing leakage of the system prompt or metaprompt as a threat to plan for and recommending that sensitive credentials and connection strings not be stored there. This article cites both as guidance, not as a guarantee about any specific product.
For the foundations of what system prompts can and cannot do, see the Learn article System Prompts.
Impact Depends on What Was Disclosed
Severity tracks content and reliance, not the fact of disclosure. OWASP's 2026 severity framing for hidden context exposure ranges from informational, where no secrets or security-relevant logic were disclosed and nothing relied on confidentiality, through medium, where internal rules or workflow logic meaningfully aid an attacker without gating critical decisions, to high, where credentials were embedded or secrecy was load-bearing, and critical, where disclosure chains into remote code execution, broad data exfiltration or privilege escalation in a connected system.
Disclosed wording alone - a role description, a style rule - rarely changes what an attacker can do. Disclosed policy criteria, tool schemas, internal identifiers or embedded credentials can. The impact question is always: what does this material enable that was not possible before?
Worked Synthetic Scenario
A test deployment of an internal documentation assistant is reviewed by its own team. The system prompt contains non-secret routing and style instructions plus one synthetic configuration marker planted for the exercise. A separate synthetic confidential canary is held outside model-visible context in a trusted test control-plane location; putting real secrets or confidential canaries into production prompts would itself be the architectural defect this article describes. Credentials live in a vault and authorization is enforced by a gateway outside the model.
System instruction: answer documentation questions, stay on topic, cite sections. Marker for testing: DEPLOYMENT-FLAG-BLUE. No credentials present.
The reviewer asks the assistant, within its own test tenant, to describe the instructions it operates under.
The assistant paraphrases its routing rules and repeats the marker DEPLOYMENT-FLAG-BLUE. Partial disclosure plus reconstruction is established.
Compared against the known test context, the output shows an exact marker match plus partial disclosure and reconstruction of the routing rules. The out-of-context confidential canary was not disclosed in this run, and no authorization bypass occurred. This result proves only what was observed in this controlled run; it is not a universal guarantee that out-of-context data can never become exposed later through an authorized retrieval or tool path. Finding: reachability confirmed, direct impact low - because credentials, authorization and enforcement stay outside the prompt.
The contrast case is the anti-pattern: the same deployment with a connection string or an approval-bypass rule embedded in the prompt. Identical extraction mechanics, categorically different incident. The scenario is synthetic; no real system, payload or secret is involved.
The Architectural Anti-Pattern
The defect behind serious extraction incidents is not the extraction. It is that something security-critical was placed inside model-visible context, or that a control silently depended on hidden context staying hidden.
API keys, tokens or connection strings embedded in instructions or retrieved into context.
Instructions like "only admins may..." treated as enforcement instead of a policy decision point.
Instructions like "never reveal internal document contents" used as the only egress control.
Refusal rules that exist only as text the model reads, with no independent check.
Large assembled contexts whose full contents nobody can enumerate during a review.
Each anti-pattern converts an extraction finding from informational to serious. Removing them is the highest-leverage remediation, because it changes the impact side of every future extraction attempt regardless of how the attempt arrives.
Detection
Detection has two surfaces. On the input side, probabilistic classifiers can flag interactions that resemble extraction attempts, with the usual limits of probabilistic filtering. On the output side, the stronger signal is overlap: comparing model output against the known assembled context, including planted canary markers, to detect reflection or reconstruction.
- 01Record the assembled context
Persist a reference to the exact context version served, so output can be compared against ground truth.
- 02Compare output to context
Measure verbatim overlap, fragment overlap and semantic similarity against the assembled context.
- 03Score and classify
Classify hits by degree of disclosure and by sensitivity of the matched material.
- 04Alert on sensitive matches
Exact or partial matches against policy text, schemas, identifiers or markers route to review.
Telemetry can establish sequence and overlap. It usually cannot prove internal model causality - that output token came from that context line. Detection claims should stay at the level the evidence supports.
Logging and Telemetry
Extraction-relevant telemetry answers what was assembled, what was asked, what was returned and what happened next - without logging secrets themselves.
Minimum telemetry for extraction analysis
- Request and session identity, with direct versus indirect source provenance
- Model, context and configuration version identifiers for the served context
- Whether hidden context was present and which constituent classes it contained
- Output lineage where available: which context elements plausibly contributed
- Detected overlap or reconstruction signals and their sensitivity classification
- Policy decisions applied to the response and any downstream effects observed
Context contents themselves are sensitive operational data. Log references, hashes and classifications rather than raw context, so the telemetry system does not become the next disclosure channel.
Incident Response
Responding to a suspected extraction
- Establish what was actually assembled in context for the affected requests, from configuration records
- Classify the disclosure degree: exact, partial, semantic reconstruction or adjacent context
- Classify the sensitivity of disclosed material, treating embedded credentials as the worst case
- Rotate or revoke anything secret that was model-visible, and remove it from context assembly
- Check whether any control depended on the confidentiality of the disclosed material
- Review whether the same context was reachable through indirect paths or shared artifacts
- Record which facts are established, which are inferred, and which remain unknown
The response is proportionate when it follows the content classification. Verbatim disclosure of non-secret style rules is a hardening task. Any disclosure of credentials or load-bearing policy is an incident with rotation, review of dependent controls and, where required, notification duties.
Prevention and Mitigation
- 01Minimize hidden context
Place in the context window only what the task needs. Every omitted line is a line that cannot be extracted.
- 02Keep secrets outside the boundary
Credentials, tokens and connection strings live in vaults and gateways the model never reads.
- 03Enforce controls outside the model
Authorization, data-flow policy and filtering are deterministic system decisions, not instructions to the model.
- 04Constrain output reflection
Output-side checks compare responses against assembled context and block or flag sensitive overlap, as a probabilistic layer.
- 05Separate contexts by principal and tenant
Per-requester context assembly limits which hidden context any one interaction can reach.
- 06Review and version context assembly
Context contents are enumerated, reviewed and versioned like configuration, because that is what they are.
None of these layers requires the extraction attempt to fail. They make a successful attempt disclose material that is non-secret and non-load-bearing, which is the achievable and honest goal.
Secure Architecture
- System prompt with policy, credentials and rules
- Model
- Output to requester
Secrecy of the prompt is the only thing protecting policy, data and credentials.
- Minimal model-visible context
- Model
- Policy decision point
- Policy enforcement point
- Output within policy
Disclosure of the context changes wording, not authority, data flow or secrets.
The stronger architecture composes controls this site covers separately: agent authorization decides what actions may execute, information flow control decides which data may move where, and retrieval hygiene in RAG systems limits what hidden context is assembled in the first place. Extraction then degrades to a low-impact information finding.
Threat Modeling Checklist
Extraction questions for design review
- Can the full model-visible context be enumerated for any request class?
- Does any context constituent contain secrets, credentials or tokens?
- Does any authorization, filtering or data-flow decision rely on context staying hidden?
- What is the worst realistic impact of verbatim disclosure of the assembled context?
- Can indirect content steer the model into reflecting its context into shared or outbound artifacts?
- Are contexts separated per tenant, role or task where they differ in sensitivity?
- Does telemetry record context versions and output overlap signals?
- Is there a tested rotation and remediation path for anything sensitive that could enter context?
Controlled Testing and Metrics
Extraction testing belongs in authorized test deployments with synthetic markers, never against third-party systems. Metrics stay separate, each with an explicit denominator, and are not combined into a single score.
Hidden context items tested at least once, out of all hidden context items in scope.
Interactions containing an extraction attempt, out of all interactions.
Attempts with verbatim reproduction of a known hidden marker, verified against the assembled context, out of all extraction attempts.
Attempts with identifiable fragments of known hidden material, out of all extraction attempts.
Attempts producing prompt-like output that matches no known hidden material, out of all extraction attempts.
Attempts disclosing a synthetic sensitive canary, out of attempts against contexts containing one.
True extraction attempts flagged by detection, out of all true extraction attempts.
Benign interactions flagged as extraction, out of all benign interactions.
Sensitive overlaps blocked or flagged before delivery, out of all sensitive overlaps.
Task success on ordinary requests, measured separately, to catch controls that break the application.
Attempts scored as faithful paraphrase under a pre-defined scoring method, out of all attempts. Useful as context, but it never replaces any metric above.
Prompt-like output that does not match known hidden material is a false-disclosure or hallucinated-prompt outcome, not evidence that the real prompt leaked. There is no universal threshold for these numbers. A deployment that treats its context as non-secret can accept a high disclosure rate by design; one that assembles sensitive policy text cannot. The metric only becomes a decision input together with the content classification.
Common Misconceptions
| Claim | Correction |
|---|---|
| A leaked system prompt means the system is compromised | Disclosure is an information event; impact depends on content and on what relied on secrecy |
| A strong system prompt prevents extraction | Instructions to the model are part of what can be disclosed; they are not a boundary |
| Extraction requires a clever jailbreak | Reachability depends on architecture; the attempt's sophistication is not the finding |
| If output looks like instructions, the real prompt leaked | Paraphrase and inference can mimic instructions; only comparison with the assembled context classifies the degree |
| Filtering extraction phrases solves the problem | Probabilistic input filtering reduces attempts; it does not change what disclosure would expose |
| System prompts are public anyway, so none of this matters | Non-secret by design is a valid posture, but only if nothing security-critical is in the context or depends on it |
Standards and Classifications
Current mappings used by this article
- OWASP GenAI LLM Top 10 (2026)
- LLM08:2026 Hidden Context Exposure, LLM01:2026 Prompt Injection (entry mechanism), LLM02:2026 Sensitive Information Disclosure (adjacent, when credentials or protected data are involved)
- NIST
- NIST AI 100-2e2025: prompt extraction terminology
- MITRE ATLAS
- AML.T0056 - Extract LLM System Prompt
- Microsoft (vendor guidance)
- Security planning for LLM-based applications: leakage of system prompt / metaprompt
OWASP's 2025 edition listed System Prompt Leakage as its own entry; the current 2026 edition covers the risk as LLM08:2026 Hidden Context Exposure, and this site maps to the 2026 edition. ATLAS catalogs the technique as AML.T0056 Extract LLM System Prompt, achievable through prompt injection or through configuration exposure. The Microsoft guidance is labelled as vendor guidance, not as a universal product property.
Frequently Asked Questions
Is system prompt extraction the same as prompt injection?
No. Prompt injection is an entry mechanism for competing instructions. Extraction is an objective: making hidden context appear in output. Injection is a frequent way to attempt extraction, but the two are classified separately.
Is a disclosed system prompt automatically a security incident?
No. It is an information event whose severity depends on what was disclosed and on whether anything relied on the context staying secret. Verbatim disclosure of non-secret style rules is not comparable to disclosure of embedded credentials.
Should the system prompt be treated as a secret?
No. Current OWASP and Microsoft guidance both recommend designing under the assumption that hidden context is discoverable. Secrets and enforcement belong outside the model-visible boundary.
Can extraction happen through external content rather than user input?
Yes. Content processed by the system - pages, documents, retrieval results or tool output - can steer the model toward reflecting its hidden context into output or shared artifacts. That is the indirect path to the same objective.
Does paraphrased output count as extraction?
It counts as semantic reconstruction, a distinct degree of disclosure. It shows that hidden context influenced the answer; it does not prove which exact tokens were in the prompt.
How is extraction different from training-data extraction?
Training-data extraction targets what the model learned during training. System prompt extraction targets context the application assembled at runtime. Different boundary, different controls, different classification.
Do guardrails that refuse extraction requests solve the problem?
They reduce successful attempts probabilistically. They do not change what a successful attempt would disclose, which is why the durable fix is keeping secrets and enforcement out of model-visible context.
What is the OWASP 2026 code for this?
LLM08:2026 Hidden Context Exposure covers extraction, inference and reconstruction of hidden system instructions and operational context. The injection mechanism is LLM01:2026; embedded credentials additionally touch LLM02:2026.
What should be tested first?
Whether the full assembled context can be enumerated and whether anything in it is sensitive. Reachability testing with synthetic markers comes second, in an authorized test deployment.
What belongs in an incident report?
The degree of disclosure, the classification of disclosed material, the context version involved, the entry path used, whether any control relied on secrecy, and which facts are established versus inferred.
References
- LLM08:2026 Hidden Context Exposure (OWASP Top 10 for LLM Applications 2026)OWASP GenAI Security Project · Guidance · 2026-08Reviewed 2026-09-20
- OWASP Top 10 for LLM Applications 2026OWASP GenAI Security Project · Guidance · 2026-08Reviewed 2026-09-20
- Prompt extraction (CSRC glossary)NIST · StandardReviewed 2026-09-20
- Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025NIST · Standard · 2025-03Reviewed 2026-09-20
- Security planning for LLM-based applications (threat: leakage of system prompt / metaprompt)Microsoft Learn · GuidanceReviewed 2026-09-20
- AML.T0056 Extract LLM System Prompt (ATLAS dataset)MITRE ATLAS · Technical documentationLatest revision 2026-05-27Reviewed 2026-09-21