Agent Evaluation
Agent evaluation measures whether an AI agent selects appropriate actions, follows constraints, uses resources efficiently, and achieves the intended outcome across a trajectory—not merely whether its final response looks plausible.
Evaluation layers
The ingested sources support several complementary layers:
- Routing and activation: Was the correct tool, skill, or specialist selected, and were irrelevant capabilities avoided?
- Trajectory: Were actions ordered correctly, grounded in observations, and recovered from when they failed?
- Artifact or outcome: Did the result satisfy functional requirements and user intent?
- Safety and policy: Did behavior remain inside permission, privacy, and risk boundaries?
- Efficiency: Were latency, token use, tool calls, and retries proportionate to the task?
- Adaptability: Did the agent respond appropriately to changed context, edge cases, and feedback?
The Google security and evaluation paper presents seven related dimensions, while the skills paper separates trigger, output, and tool-trajectory evaluation. Security and evaluation source record (pp. 27–39) · Agent Skills source record (pp. 15–24)
Methods
No single method covers the full failure surface. Production suites can combine:
- deterministic unit, integration, and acceptance tests;
- static analysis and schema validation;
- recorded trajectory assertions;
- adversarial and perturbation tests;
- model-based judges with calibrated rubrics;
- human review for intent, quality, and high-stakes decisions;
- online monitoring, user corrections, and incident-derived regression cases.
OpenAI recommends establishing an evaluation baseline before optimizing model cost. Anthropic recommends measuring whether added agentic complexity demonstrably improves outcomes. OpenAI source record · Anthropic source record
The spec-driven development paper connects reviewed acceptance criteria, generated tests, policy checks, and human review to coding-agent release decisions. Spec-driven development source record (pp. 26–35)
Unit of evaluation
A model score is not an agent score. Context assembly, tool definitions, permissions, retries, skill routing, memory, and stopping behavior can alter results. The evaluated unit should therefore include the relevant harness and deployment configuration. Agent Skills source record (pp. 21–31)
ReAct illustrates why trajectories matter: actions and observations can expose grounding and recovery behavior that a final answer hides. Its reported benchmark gains remain task-specific. ReAct source record
Observability boundary
Evaluation requires evidence about inputs, selected context, tool calls, observations, state transitions, and termination. However, more tracing can expose private data, prompts, credentials, or sensitive reasoning artifacts. Evaluation telemetry should be minimized, access-controlled, and retained according to explicit policy. Security and evaluation source record (pp. 24–39)
Judge limitations
Model-based judges scale qualitative review but can be unstable, biased toward surface features, or correlated with the system under test. They should be calibrated against human ratings, tested for ordering and prompt sensitivity, and paired with deterministic checks where ground truth exists. Agent Skills source record (pp. 17–24)
For evolving tasks, the intent rubric must be updated from the full interaction rather than only the opening request. Humans can be the final authority on intended behavior while still being unable to validate every technical property, so functional, security, and policy evidence must complement human judgment. Security and evaluation source record (pp. 28–35)
Tests generated by the same agent that produced an implementation can share its mistaken assumptions. Independent tests, static checks, separate reviewers, or diverse judges reduce—but do not eliminate—this correlated-error risk. Spec-driven development source record (pp. 26–35)
Evaluation lifecycle
Analysis: A useful loop is:
- define success and unacceptable failure;
- collect representative and adversarial cases;
- measure baseline behavior;
- change one model, prompt, tool, skill, or policy component;
- compare quality, safety, latency, and cost;
- inspect failures and add regression cases;
- monitor production drift and repeat.
Evidence status
Confidence is medium for this layered methodology because several practical sources converge on it. Confidence is lower for any universal threshold or judge metric; those require domain-specific validation and independent evidence.