What to actually measure when your agent "works"

“It works” is a demo result, not a measurement. An agent is a trajectory, not a function, and grading only the final answer throws away most of what decides whether it’s reliable. Here’s a five-layer scheme for what to measure — outcome, trajectory, cost, failure class, and stability under nondeterminism — with a small harness that computes it.

July 7, 2026 · 10 min · 2112 words · Loop & Retry