Agentic AI · Agent evaluation

Evaluating a chatbot means grading one answer. An agent has a whole trajectory: the tool calls, their arguments, retries and the final result. What I’d measure:

metric	question
task success	is the final answer or end state correct?
tool-call accuracy	right tool, valid arguments?
efficiency	steps, tokens or cost, time
safety	did it attempt anything outside policy?
failure honesty	when it fails, does it say so?

Public benchmarks focus on task success in realistic settings: GAIA (Mialon et al., 2023, summarised on this blog in Nov 2023) with questions needing browsing and tools, SWE-bench (Jimenez et al., 2023) with real GitHub issues, and AgentBench (Liu et al., 2023) across several environments.

I scored four recorded runs of the same task against a reference trajectory. The runs are hand-written examples, not real model logs.

REFERENCE_TOOLS = ["search_notes", "subtract"]
EXPECTED = "1"
runs = {
    "clean":       {"calls": [("search_notes", {"q": "BERT"}), ("subtract", {"expr": "2019-2018"})], "answer": "1"},
    "wasteful":    {"calls": [("search_notes", {"q": "BERT"}), ("search_notes", {"q": "BERT"}),
                              ("search_notes", {"q": "GPT-2"}), ("subtract", {"expr": "2019-2018"})], "answer": "1"},
    "lucky_guess": {"calls": [], "answer": "1"},
    "bad_args":    {"calls": [("search_notes", {"q": "BERT"}), ("subtract", {"expr": "2019 minus 2018"})], "answer": "error"},
}
ALLOWED = {"search_notes", "subtract"}

def valid_args(tool, args):
    return tool != "subtract" or all(c in "0123456789- " for c in args.get("expr", ""))

print(f"{'run':<12} {'success':>7} {'tools ok':>8} {'args ok':>7} {'steps':>5} {'extra':>5} {'policy':>6}")
for name, r in runs.items():
    tools = [t for t, _ in r["calls"]]
    success = r["answer"] == EXPECTED
    tools_ok = all(t in tools for t in REFERENCE_TOOLS)
    args_ok = all(valid_args(t, a) for t, a in r["calls"])
    extra = max(0, len(tools) - len(REFERENCE_TOOLS))
    policy = all(t in ALLOWED for t in tools)
    print(f"{name:<12} {str(success):>7} {str(tools_ok):>8} {str(args_ok):>7} {len(tools):>5} {extra:>5} {str(policy):>6}")

Output:

run          success tools ok args ok steps extra policy
clean           True     True    True     2     0   True
wasteful        True     True    True     4     2   True
lucky_guess     True    False    True     0     0   True
bad_args       False     True   False     2     0   True

“lucky_guess” passes on final-answer accuracy alone, but it never used a tool, so nothing grounds the answer. That’s the case for trajectory-level checks.