A practical first evaluation for an AI agent

A useful first agent evaluation starts with a narrow promise. For example: given an approved product description, prepare a draft knowledge-base article without publishing it. That promise is easier to test than a broad goal such as improving support productivity.

Anthropic’s evaluation guide distinguishes a task, repeated trials, the transcript of an attempt, and the final state of the environment. It also describes code-based, model-based, and human grading. OpenAI’s workflow-evaluation documentation recommends using traces to investigate behavior and repeatable datasets to compare changes. The method below is an editorial starter design built around those distinctions.

1. Write a result you can inspect

For the knowledge-base example, a successful run creates one draft in the correct collection, preserves approved facts, and leaves publication disabled. A natural-language message claiming success is not sufficient evidence. The test needs to inspect the saved article and its actual publication state.

Separate the useful outcome from unacceptable side effects. A draft containing the right information still fails the boundary check if it overwrites another article. Keep the quality and boundary results visible as separate fields, so a high writing score cannot hide an unauthorized action.

2. Build a small, varied case set

Start with a routine request, a missing input, a conflicting source, and an interrupted tool call. Add a case where the agent must stop because the requested operation exceeds its permissions. Choose examples from the work the product is actually intended to perform.

For each case, record what the agent is allowed to see and change. Use synthetic or properly de-identified data in the test environment. Do not give the test agent production credentials merely to make the exercise feel realistic.

Write the expected disposition before running the system. Missing evidence might require a clarification. A temporary service failure might permit a retry. A denied publishing permission should not be treated as a technical inconvenience to work around.

3. Choose the right evidence for each claim

Use a direct state check for facts such as whether an article exists, whether it is a draft, and whether another record changed. Use a small editorial rubric for qualities such as accuracy, readability, and appropriate uncertainty. A reviewer should be able to explain a failure with a specific example.

If you use a model to grade prose, first compare its decisions with a few careful human judgments. Include borderline examples. Record disagreements rather than averaging them away immediately; a disagreement may reveal that the product requirement is still ambiguous.

4. Repeat the trial without changing the question

Run more than one attempt for cases where behavior varies. Reset the environment between attempts so an earlier draft, cached answer, or permission change does not quietly influence the next result. Record the configuration with each run.

A compact result sheet can contain the task identifier, outcome check, boundary check, quality assessment, elapsed time, and a link to the trace. Note separately when a failure comes from the test setup. An unavailable fixture is not automatically a model defect.

5. Read failures before tuning prompts

Inspect where the run first departed from the intended behavior. Did the agent lack a fact? Was the tool description ambiguous? Did a retry duplicate a write? Did the grading rule reject a valid alternative? Each diagnosis suggests a different repair.

Change one meaningful part of the system, then rerun the relevant cases and the cases that previously worked. Keep a small set of examples out of routine prompt tuning so the team can check whether an improvement extends beyond familiar tasks.

6. Decide what the evidence permits

A first evaluation might support a limited pilot with drafts and human review. It may not support automatic publication. State that boundary explicitly. Identify who will inspect pilot failures and how the agent can be stopped if its behavior changes.

As real use reveals new cases, add them to the suite with a note explaining the failure they protect against. The evaluation becomes a maintained description of the product’s promises, rather than a score collected once before launch.