Step 1 of 6 · Testing - practitioner kit
Test Plan Template
Defines identity, authorization, threat model, instrumentation, case selection and closure for one authorized assessment.
Use this template
A template is not evidence. Fill in sanitized identifiers only.
Purpose
Fixes what is tested, under whose authority, against which versions and with which evidence before any stimulus is introduced.
When used: Before the first run, and again whenever scope, versions or authorization change.
A. Assessment identity
Every later record refers back to these identifiers and versions. If a version changes mid-assessment, record a new plan revision rather than editing history.
| Field | What to record |
|---|---|
| Assessment ID | Stable identifier, for example ASM-2026-014. Reused in every run, evidence and finding record. |
| Title | Short descriptive name of the system and objective. |
| Owner | Accountable person on the system side who receives results. |
| Technical lead | Person responsible for execution quality and evidence handling. |
| Date / window | Planned start and end, including time zone and any freeze periods. |
| Environment | Sandbox, staging or faithful test environment; state how it differs from production. |
| System / application version | Build, commit or release identifier of the application under test. |
| Model / provider / version | Model identifier and provider version string as reported by the provider or gateway. |
| Prompt / template version | Version of system prompts, templates and orchestration instructions. |
| Policy version | Version of authorization, data-flow and approval policies that the controls enforce. |
| Tool / catalogue / config version | Tool registry, MCP catalogue snapshot, connector and retrieval configuration versions. |
B. Authorization and governance
Testing without written authorization is out of scope for this toolkit. The plan records the authorization; it does not replace it.
| Field | What to record |
|---|---|
| Authorizing owner | Named person with authority over the systems in scope. |
| Written authorization reference | Ticket, contract or memo identifier. Store the document itself in the agreed location. |
| In-scope systems / components | Applications, agents, connectors, corpora and tools that may be exercised. |
| Out-of-scope systems | Explicit exclusions, including production data, third-party services and real external destinations. |
| Permitted actions | Benign operations allowed, for example changing a label on a fictional sandbox record. |
| Prohibited actions | Destructive operations, real exfiltration, credential use outside test accounts, evasion of monitoring, testing of real third parties. |
| Test identities / accounts | Dedicated or attributable test principals, their roles and what they can reach. |
| Stop conditions | Events that halt testing immediately, such as unexpected access to real data or an effect outside the sandbox. |
| Emergency contact / escalation | Who to call, in which order, and how fast. |
| Data handling / retention | Where evidence is stored, who can access it, how long it is kept and how it is deleted. |
C. Objective and threat model
| Field | What to record |
|---|---|
| Business / security objective | The decision this assessment should inform, stated as a question. |
| Relevant testing domains | Domains from the Testing methodology that apply, for example RAG, agent boundaries, flow and egress. |
| Attack surfaces | Channels through which untrusted content can reach model-visible context. |
| Trust boundaries | Where content changes trust level, and which component decides that. |
| Principals | Users, service identities and agents whose authority actions run under. |
| Resources | Records, files, mailboxes or stores that actions can read or change. |
| Tools | Callable capabilities, their parameters and their side effects. |
| Destinations | Where data can leave: network endpoints, messages, rendered links, files. |
| Sensitive data classes | Classes the controls must protect; use fake data of each class in the sandbox. |
| Expected deterministic controls | Application-owned controls that should decide independently of model output, such as authorization, flow policy, egress allowlist and approval binding. |
D. Evidence and instrumentation plan
Decide before testing which records are authoritative. If a required source is missing, plan for Inconclusive rather than substituting model narration.
| Field | What to record |
|---|---|
| Required logs | Application, orchestration, control decision, tool gateway and provider audit logs needed for each evidence stage. |
| Correlation IDs | How a single run is traced across components: request, trace or conversation identifiers. |
| Timestamps / time synchronization | Time source, time zone and acceptable clock skew between systems. |
| Provider / application audit source | System of record for downstream state, for example the sandbox record store audit log. |
| Evidence retention location | Access-controlled storage for raw and normalized evidence. |
| Authoritative downstream-effect evidence | What exactly proves stage 7 for this system, for example a state read from the record store, not a model statement. |
E. Test selection
| Field | What to record |
|---|---|
| Selected PI-TST cases | Case IDs from the library, or local case IDs built with the Test Case Template. |
| Local adaptations | Changes to objective, oracle, fixtures or pass criteria, with rationale. |
| Positive controls | Benign in-scope operations that must succeed so a deny-all implementation cannot pass. |
| Number of controlled runs | At least three fresh-session runs per stimulus where model influence is part of the objective. |
| Sequencing / dependencies | Order of baseline, positive control, stimulus and variants; resets between runs. |
F. Reporting and closure
| Field | What to record |
|---|---|
| Finding owner | Who writes and maintains finding records. |
| Remediation owner | Who owns fixes on the system side. |
| Retest criteria | Which evidence closes a finding, recorded in the Regression Retest Record. |
| Residual-risk owner | Who accepts remaining risk in writing. |
| Sign-off | Names, dates and plan revision that were signed. |
Minimum viable plan
For small assessments, these fields are the floor. Anything below this is not a plan an independent reviewer could check.
| Field | Why it cannot be skipped |
|---|---|
| Assessment ID and versions | Without versions, results cannot be reproduced or retested. |
| Written authorization reference and scope | Defines what may be touched at all. |
| Prohibited actions and stop conditions | Keeps the test safe when something unexpected happens. |
| Objective and expected deterministic control | Gives each case a decidable oracle. |
| Authoritative evidence source for downstream effect | Prevents model narration from being treated as proof. |
| Selected cases with positive controls | Prevents deny-all implementations from passing. |
| Owner for findings and residual risk | Ensures results lead to a decision. |
How to use this template
- Fill sections A and B first and get sign-off before designing stimuli.
- Derive section C from the real architecture, not from a generic diagram.
- Treat section D as a precondition: if you cannot observe a stage, mark the affected case outcomes as at risk of Inconclusive now.
- Revise the plan, with a new revision number, whenever a model, prompt, policy or catalogue version changes.
Common mistakes
- Writing the plan after testing has started.
- Listing 'the model' as the control instead of an application-owned decision point.
- Omitting positive controls, so a blocked everything result looks like success.
- Naming no authoritative downstream source and later relying on chat transcripts.
- Treating the plan as proof that the assessment was authorized or complete.