Data Provenance

Security Objective

Data provenance is the record of where content came from and how it became what a model saw or a system stored. It captures the source system, the resource and version, the tenant and principal involved, the supplier and author where known, and each transformation between the original artifact and the model-visible or persisted result.

Its security value is traceability. With complete lineage, a team can explain which source shaped a model proposal, find every summary, cache entry, index record or memory derived from a compromised source, and reconstruct the evidence behind a flow or authorization decision. Other controls consume that evidence: information flow control uses provenance and classification to decide flows, and agent authorization uses principal and resource attributes to decide execution.

Ownership Boundaries

Provenance does not decide whether a source is trusted, whether a user may access content, whether data may flow, or whether an action may execute. If a system uses the provenance record itself as an allow or deny switch, that decision still belongs to a policy, and that policy should be named and owned. Provenance feeds spotlighting / data marking, which decides how content is presented to the model.

Provenance Is Not Trust

The most common misuse of provenance is to treat a known origin as a trust verdict. Five distinctions prevent that.

Trust Model

Payload and provenance metadata are separate security objects. Human-readable payload never becomes system metadata, and a system component that faithfully processes content never upgrades that content's trust. The flow below alternates between the two planes so the distinction stays visible.

Read the flow as two parallel planes. Payload nodes (source content, captured payload, derived artifacts, model-visible text, proposals, effects) stay variable however many trusted components touch them. Metadata nodes (envelope, lineage records, derived-artifact lineage) are produced by system components from authoritative inputs such as connector configuration, identity services and storage keys, not parsed from payload text. A parser is a system component, but its output is a derived payload with the same untrusted character as its input.

Provenance Versus Neighbouring Properties

Source Identity Versus Author Identity

The component that delivered content is often not the party that wrote it. A search tool returns pages written by strangers. A ticketing API returns fields edited by customers. A mail connector returns forwarded messages quoting several people.

Core Provenance Fields

Authoritative Metadata Plane

Provenance used for security decisions must live outside attacker-editable and model-authored payload text. A structured envelope carries metadata alongside the payload; the payload never populates the envelope.

Transformation Lineage

Each step between original artifact and model-visible text should leave a lineage record naming the input, the output, the component and its version.

One-to-Many and Many-to-One Derivations

Lineage is a graph, not a chain. One document yields many chunks; many chunks yield one summary; one summary may be cached, embedded and written to memory. Records should link each derived artifact to all its inputs and each input to all its derivatives, so that queries work in both directions: from a model-visible span back to sources, and from a source forward to everything derived from it.

Model-Visible Span Attribution

The most useful record for incident work is the mapping from each span or field in the assembled context back to its artifact, transformation chain and source span. When a model proposal is questioned, investigators can then see exactly which retrieved text, tool field or memory entry was present, and where it came from.

Hashes, Signatures and Attestation

A signed document from a verified partner can still contain adversarial text. A digest that matches the captured version proves only that the bytes are the same. Use these mechanisms to protect lineage records and to detect change; never let them substitute for flow or action policy.

Tool and Agent Provenance

For tool results, the tool identity is the delivering component, not the author of the payload. Preserve the final resource, record and field contributors where the tool exposes them.

Across agents, the sender agent's identity is not enough. Preserve the sources the sender consumed and the transformations it applied, so that a receiving agent's context can be traced past the handoff. There is no transitive trust: content forwarded by a trusted agent keeps the provenance and character of its original sources.

Persistent and Derived Artifacts

Summaries, caches, index records, memory entries and generated records outlive the request that created them, which is why persistent injection is hard to clean up without lineage. Each should keep links to its inputs, the producing component and version, and the time of creation.

Embeddings and index entries are derived artifacts with lineage to source, chunk, index and embedding model version. A model normally sees the retrieved text, not the vector, and vectorization does not sanitize semantic content; the retrieved chunk carries the same character as its source.

Mixed-Source Derivations

An artifact built from several sources keeps the full contributor set and each input's classification reference. It is never silently collapsed to one label, and certainly not to the most trusted input's label.

How labels are combined for a given decision is not a provenance rule. The consuming policy, for example information flow control, decides how classifications join for a particular source and sink. Provenance's job is to make the full set available.

Missing and Ambiguous Provenance

When origin or lineage is missing, record it as unknown. Never reconstruct provenance from model output or attacker-editable fields, and never fill gaps with a default source.

Whether unknown provenance blocks a flow or action is a downstream policy decision. Where a protected policy requires fail-safe handling, that handling lives in information flow control or authorization, not in the provenance record itself.

Lineage Record Integrity

Lineage records are only useful if they cannot be quietly rewritten. Append-only or immutable storage is one optional implementation pattern; access-controlled records with change audit, or integrity-protected logs, are others. No storage technology is universally required.

Immutability preserves records. It does not create semantic trust in the payload those records describe.

Versioning, Quarantine and Revocation

When a source changes, is quarantined or has its access revoked, lineage lets teams enumerate the derived artifacts and affected requests. Provenance itself does not delete or invalidate those derivatives. A separate, owned process must act on the enumeration: invalidate caches, rebuild indexes, retract memory entries and re-evaluate stored summaries.

Telemetry and Correlation

Request, task and session correlation IDs connect provenance to runtime events: which request assembled which context, which proposal followed, which policy decision and effect resulted. Correlation IDs should be generated by the system, carried in metadata, and never taken from payload.

Privacy, Minimization and Retention

Lineage metadata can itself be sensitive: principal IDs, tenant names, file paths and source spans. Collect enough to reconstruct security decisions and support cleanup, avoid uncontrolled duplication of payload inside lineage records, apply the same access control as the data described, and set retention that balances forensic needs against minimization.

Incident Response and Lineage-Aware Cleanup

Worked Synthetic Scenario

All names and values are synthetic. A support workflow for tenant "Tenant-A" ingests a customer-submitted document, "contract-draft.pdf" version 3, uploaded by contributor "customer-417". A parser at version 2.4 normalizes it; the result is split into twelve chunks. Chunk 7 contains instruction-like text addressed to an assistant. Later, a summarizer produces a case summary from chunks 5 to 8; the summary is cached, embedded and written to case memory. A model proposal to send a document externally cites the summary.

Review flags the proposal. Investigators start from the model-visible span and follow lineage: span, then summary, then chunk 7, then parser 2.4, then contract-draft.pdf version 3, then contributor customer-417 in Tenant-A. Forward lineage from the document lists the twelve chunks, the summary, its cache entry, its index record and the memory entry. Correlation IDs enumerate the requests that used any of them. Owning processes quarantine the document, invalidate the cache, remove the index record, retract the memory entry and mark the requests for review.

Testing and Metrics

Measure each property on its own with an explicit denominator. There is no composite score; benign task utility is reported separately.

To practice judging provenance records as evidence, try the Data Provenance lab.

Implementation Checklist

Standards and Sources

On source use: the NIST glossary defines provenance as the chronology of the origin, development, ownership, location and changes to a system or component and associated data; it is general terminology, not AI-specific implementation guidance. W3C PROV-DM supplies a general vocabulary of entities, activities, agents and derivations; this site does not claim to implement formal W3C PROV. Microsoft's retrieval hygiene guidance states that secure retrieval requires source provenance, permission-aware indexing and validation at every read and write, and recommends retaining read and write lineage; the Agent Safety page describes trust boundaries for externally influenced data. Both are vendor guidance.

Frequently Asked Questions