What is an AI agent?
An AI agent is an application pattern in which a model's output is used to choose and trigger actions, often over several steps. The application gives the model a task, relevant context and a description of available capabilities, interprets the model's structured output as proposed actions, executes some of them and feeds the results back.
"Agent" is not a magic model capability and not one universally standardized primitive. Products use the word for anything from a single tool call to long-running multi-step systems. What they share is the step this article is about: model output crosses from text into action.
This builds on System Prompts and Trust Boundaries and Model Context: model-visible context can influence the output, but trust, authority and security-relevant facts are established by application-side state; the model's output remains a variable proposal.
What is tool or function calling?
Tool calling (also called function calling) means the model emits structured data, such as a tool name and arguments, that the application may interpret as a request to invoke a capability. The model itself does not run the tool. Application code decides whether to act on the request, how, and with which credentials.
The agent loop
| Stage | What it is | Trust character |
|---|---|---|
| Task or request | User request is variable input; identity, scope and task envelope are application-owned metadata | Mixed: user request is variable input; task metadata is application-owned |
| Context assembly | Instructions, conversation, retrieved content, earlier results, tool descriptions | Model-visible variable content plus trusted application metadata kept outside the model |
| Proposal | The model's text or structured tool call | Model output, variable data |
| Schema validation | Checks shape and types of the proposed call | Structural contract only |
| Authorization and policy | Decides whether this principal may perform this exact action now | Deterministic application decision |
| Approval when required | Authenticated human decision on the exact action | External evidence tied to one action version |
| Execution | The application invokes the capability with scoped credentials | Capability invocation |
| Result | Observed execution state plus content returned by the tool | State is observed; returned content is variable data |
| Re-entry into context | The result is added to context for the next step | Provenance preserved |
| Next proposal | The model proposes a further step or an answer | Again variable data |
Every arrow in this loop is a transition the application controls. Security decisions belong at those transitions, not inside the model.
Model proposal vs application decision
Keep three things apart: what the initiating user asked for, what the model proposed, and what the application decided to execute. "The model wanted it" does not equal "the user requested it". A proposal can be influenced by any model-visible content, including third-party text, as described in AI Agent Prompt Injection.
Model-authored claims inside a proposal, such as "the user already approved" or "this recipient is internal", are content to validate, never facts for the permission decision. This is the core rule of Agent Authorization.
Tool availability is not permission
Showing a tool or schema to the model means the model can propose using it. It does not mean every caller, session or task may use every action, resource or parameter that tool supports. Availability describes what can be requested; permission is decided per call.
Reducing the set of tools exposed to a task is still useful: it narrows what the model can propose and reduces confusion. It complements authorization but does not replace it.
Tool schemas: what they validate and what they do not
| A schema can check | A schema does not establish |
|---|---|
| Required fields are present | That the caller may perform the action |
| Types, formats and enumerated values | Where the argument values came from |
| Value ranges and lengths | Business-policy compliance |
| A well-formed call shape | Tenant isolation or resource ownership |
| Rejection of unknown fields | Semantic safety of the request |
Schema validation is part of output validation: it checks shape and contract. Sink-specific encoding, escaping or parameterization for the system that consumes the values remains a separate step. After validation, model-authored arguments are still variable data.
Identity: user, agent and connector
Authentication and authorization are distinct. A valid, authenticated connector or credential proves who is calling; it does not make arbitrary model-proposed actions authorized.
The agent's identity and the initiating user's identity are also distinct. Common patterns include:
- Act as the user: calls run with the user's delegated permissions, so the agent can reach no more than the user can.
- Service identity: the agent runs under its own identity; the application must then still check what the initiating user is entitled to.
- Delegated, attenuated capability: the agent receives a narrower, task-scoped grant derived from the user's permissions.
None of these is a universal design. Each requires the application to record who initiated a task and under which identity each call executes.
Runtime authorization and policy inputs
A runtime authorization decision should bind:
What the decision binds
- Principal and session: the initiating user and the executing identity.
- Action or tool: the exact operation requested.
- Resource: the specific record, file, account or system.
- Parameters and destination: values, recipients, amounts, targets.
- Current state: status of the resource and task at decision time.
- Approval or delegation, where policy requires it.
These facts come from application-side records, not from model text. Destinations and data movement are additionally governed by Information Flow Control.
Least privilege and capability scoping
Least privilege limits the blast radius when model behavior goes wrong. Scope credentials to the task, prefer read-only where possible, restrict reachable resources and destinations, and keep grants short-lived. Then even a fully influenced proposal can only reach what the scope allows.
Human approval and step-up controls
For selected consequential actions, policy can require a human approval. Approval is authenticated evidence and consent bound to one concrete action version; it is an additional gate, not a replacement for authorization. It never overrides a deterministic deny and never widens credentials.
A stale approval for an earlier version, a generic "allow this tool" confirmation, or model-authored text claiming approval is insufficient. No response or a timeout for a consequential action means no execution. See Human-in-the-Loop.
Tool output returns as variable data
A tool result goes back into model context. Unless a field is independently established by the application, such as an execution status code, the returned content is variable data. A trusted tool does not make every returned field trusted or instruction-authoritative: a trusted API can faithfully return text written by anyone.
Content returned by tools can therefore influence later proposals. This separate risk is described in Tool Output Injection. Preserve provenance when results re-enter context so the application, not the model, decides how each value may be used.
Multi-step plans, plan drift and tool-chain risk
A chain of individually permitted steps can produce an outcome no single step reveals, for example reading sensitive data in one step and moving it in another. Plan drift describes a proposal moving away from the task the user started; tool-chain analysis looks at data and effects across a sequence of calls.
Both are monitoring and detection concepts, supported by Prompt Injection Detection. They surface evidence for review; they are not permission systems. Authorization and flow control still decide each call. In multi-agent systems, one agent's output becomes another agent's input and needs the same treatment.
Retries, idempotency and uncertain execution state
A call can time out or return no observable result. An unknown result is not the same as a failure. For consequential operations, blindly repeating the call can perform the action twice.
Before retrying, check the operation state, use idempotency keys where the target supports them, and record that the outcome was uncertain. Let the application, not the model, decide whether a retry is safe.
Proposed, authorized, executed, succeeded
| State | Meaning | What it does not mean |
|---|---|---|
| Proposed | The model emitted a call | That anyone may or will execute it |
| Authorized | Policy permitted this exact call | That it ran or will succeed |
| Executed | The application invoked the capability | That the operation completed |
| Succeeded | The operation was observed to complete | That the returned content is trustworthy |
Synthetic examples
A support session for customer C-100 asks about order status. The model proposes lookup_order for an order owned by C-100.
Schema validation passes; authorization confirms the order belongs to the session's customer; the read executes and the status is summarized. A clean allow path.
In the same session the model proposes lookup_order for an order owned by customer C-200.
The call is well-formed and the tool is available, but authorization rejects it because the resource does not belong to the session's customer. The denial is logged.
A retrieved document mentions reports@partner.example.test. The model proposes send_email with a summary to that address.
Destination policy checks the recipient against the application's allowed destinations. The address came from content, not from the user or an approved list, so the send is rejected unless independently allowed.
A fictional refund of 20 credits on order O-7 times out with no observed result.
The application queries the operation state using the idempotency key before any retry. The refund had completed, so no second refund is issued.
A ticket tool returns a description field containing [synthetic instruction-like text].
The field is kept as ticket content with its source recorded. It does not gain instruction authority, and any follow-up proposal is authorized on its own merits.
Logging, telemetry and audit trail
For consequential actions, keep auditable records outside the model:
Record per call
- Initiating principal and executing identity.
- Task or request identifier and correlation ID.
- Exact proposed call and parameters.
- Authorization decision and the policy that produced it.
- Approval event, if any, with reviewer and action version.
- Execution state and result.
- Provenance of inputs that shaped the proposal.
- Timestamps for each state transition.
Logs support review and response. They are evidence, not enforcement.
Common misconceptions
Statements that do not hold
- If the model called the tool, the user wanted it.
- A tool shown to the model may be used for any resource.
- Schema validation makes a call safe.
- An authenticated connector authorizes any action it can technically perform.
- Results from a trusted tool are trusted instructions.
- A timeout means the action did not happen.
- Approval of a tool once covers later calls.
Practice
The labs train each transition with fictional scenarios: Agent Authorization for the permission decision, Tool Output Injection for results re-entering context, Plan Drift Detection and Tool-Chain Analysis for multi-step risk, and Human Approval for approval validity.
Standards and sources
How this article uses sources
- OWASP
- LLM03:2026 Excessive Agency - excessive functionality, permissions and autonomy, Top 10 for Agentic Applications 2026 - agent-specific risk framing, GenAI LLM Top 10 2026 - prompt injection as a named risk
- NIST
- AI 100-2e2025 - prompt-injection terminology, including indirect injection
- Microsoft
- Least privilege for AI agents and the AI agent shared responsibility model, Indirect prompt-injection guidance - layered mitigations
Exact authorization binding, the four execution states, idempotent retries and the audit record are presented as architecture guidance, not as requirements quoted from a standard.
Frequently asked questions
Does the model execute tools itself?
Is a valid tool call an authorized tool call?
Should the agent use the user's identity or its own?
Can human approval replace authorization?
Are tool results safe to follow?
What should happen after a timeout?
Does plan drift detection block actions?
References
- LLM03:2026 Excessive AgencyOWASP GenAI Security Project · GuidanceReviewed 2026-09-26
- OWASP Top 10 for Agentic Applications 2026OWASP GenAI Security Project · GuidanceReviewed 2026-09-26
- Least privilege for AI agentsMicrosoft Learn · GuidanceReviewed 2026-09-26
- AI agent shared responsibility modelMicrosoft Learn · GuidanceReviewed 2026-09-26
- OWASP GenAI LLM Top 10 2026OWASP GenAI Security Project · GuidanceReviewed 2026-09-26
- Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025NIST · Standard · 2025-03Reviewed 2026-09-26
- Defend against indirect prompt injection attacksMicrosoft Learn · GuidanceReviewed 2026-09-26