How do you evaluate an agent?

Answer

Start from what success means for the task, expressed as something you can check without a human reading the output.

Prefer end state checks over trajectory checks. For a coding agent: do the tests pass, does it compile, does the diff apply. It does not matter which path it took, and grading the path punishes valid alternative approaches.

Build a fixed evaluation set. Real tasks with known good outcomes, versioned alongside the code. Twenty realistic cases beat a thousand synthetic ones. Without a fixed set you cannot tell whether a prompt change helped, and you will ship regressions.

Measure the things that actually bite:

• Success rate on the task
• Steps taken, since more steps means more cost and usually more drift
• Token cost per task
• Latency
• Rate of destructive or out of scope actions

Run each case several times. Agents are nondeterministic. A single pass tells you almost nothing. Report a success rate over five or ten runs, and treat a change from 8/10 to 9/10 as noise unless the sample is larger.

Use an LLM judge only where programmatic checks cannot reach, such as writing quality. Judges are biased toward verbose and confident output, so validate the judge against human labels before trusting it.

Common wrong answer

Evaluating by reading a few outputs and deciding they look good. This is how teams ship changes that improve the cases they looked at and degrade the ones they did not. It also cannot detect a regression, because there is nothing to compare against.

Another mistake is grading the reasoning trace. Two agents reaching a correct result by different routes are both correct, and rewarding a particular route encourages overfitting to your idea of the solution.