Guided security lab

Poisoned Knowledge Base

A guided corpus-integrity exercise: separate retrieval relevance from source integrity, writer permission from policy authority, and verified knowledge updates from poisoned or stale records, using application-side provenance and approval state.

Conceptual simulationIntermediate15 minutes2026-09-26

Conceptual simulation

This exercise uses authored, deterministic examples with fictional names and neutral synthetic claims. It runs no retrieval, embeddings, index, model or tools, and it does not predict how any real model behaves.

Step 3 of 63/6

Learning objectives

Separate retrieval relevance from corpus and source integrity.
Explain why write permission is not authority to define organizational truth or policy.
Rely on application-side provenance, source-of-truth designation, approval state, version and integrity state instead of model-visible self-description.
Treat conflicting, missing or quarantined integrity metadata conservatively, and invalidate derived artifacts after a source is quarantined.
Recognize a verified knowledge update and allow a bounded answer, while keeping knowledge poisoning distinct from prompt injection.

Prerequisites

RAG Prompt Injection lab (recommended)
RAG Prompt Injection attack article (optional)

Choose a scenario

Each fictional scenario shows retrieved content the model sees and, separately, the provenance, approval and integrity facts the application records. Integrity decisions belong to the second column.

Highly relevant claim with unverified integrityA newly ingested knowledge item matches the question closely and states a synthetic policy fact. It contains no instruction-like text, but its source integrity and approval state are unverified.Legitimate writer, but no policy authorityA contributor may add and edit ordinary wiki pages. Their new page states a synthetic security-policy change, but they are not authorized to publish canonical policy.Newer, more relevant candidate conflicts with the canonical sourceA recent, highly ranked document contradicts the verified canonical version. The model proposes following the newer candidate.Verified knowledge update through the approved workflowA policy owner published a new version through the source-of-truth workflow. Provenance, approval, supersession and ACL are intact, and the user asks a read-only question.Stale derivative of a quarantined sourceA source was quarantined after an integrity issue. An old summary cache built from it still exists and is retrieved.

Knowledge path and trust flow

Knowledge moves from sources through ingestion, integrity checks, indexing and retrieval before the model sees it. Components are system; documents, derivatives and the model proposal stay variable.

New knowledge itemVariable content->
Ingestion and integrity checkSystem component->
IndexSystem component->
RetrieverSystem component->
ModelSystem component->
Proposed answerVariable content->
Answer integrity policySystem component

Model-visible retrieved content

Retrieved text the model reads. Bracketed text marks a neutral synthetic claim, not real guidance.

Retrieved item kb-204, tagged unverified
[synthetic factual claim] Travel reimbursements are now paid within two days.

Application-side provenance, approval and integrity

Recorded by the ingestion pipeline, the source-of-truth registry and the application outside model text.

Relevance: Top result for the question; relevance is not integrity evidence
Source integrity: Unverified: origin not matched to a registered source
Approval state: Not approved; no reviewer recorded
Answer policy: Only verified, approved knowledge may be stated as fact