概述
That covers whether it finished the task, whether the path it took was sensible, whether it used tools correctly, what it cost, and whether it stayed safe. Agents act over many steps and have side effects, so checking a single answer against a reference is not enough, and good evaluations usually run the agent inside a controlled environment and inspect the final state.
深入探討
Agent evaluation looks at several things. Task success asks whether the goal was actually achieved, ideally checked by looking at the environment afterward: tests pass, the right record exists, the file has the right contents. Trajectory quality asks whether the steps were reasonable, without loops, pointless calls or lucky guesses. Tool-use correctness checks whether the agent picked the right tools with valid arguments and handled errors. Cost and latency track tokens, calls and wall-clock time. Safety checks whether the agent avoided harmful or unauthorized actions and resisted prompt injection. Environment-based benchmarks have become a leading approach. SWE-bench, from Princeton researchers, asks agents to resolve real GitHub issues and checks the results with the project's tests, and SWE-bench Verified is a human-checked subset. WebArena provides self-hosted websites for browsing tasks. OSWorld tests agents operating full computer desktops. GAIA poses questions that need multi-step research and tools. The tau-bench suite simulates users and domain policies, and it introduced a pass^k metric that asks whether an agent succeeds on every one of k repeated attempts. Grading uses three main kinds of checks: code-based checks on outcomes, which are precise and cheap; model-based graders that score transcripts against a rubric, which are flexible but need to be checked against human judgment; and human review, which is the most trustworthy and the slowest. A common mistake is to trust public leaderboards alone. Benchmarks can leak into training data, become saturated, or fail to resemble your own workload. Another mistake is treating one run as the truth, since agents are nondeterministic and small differences in success rate can be noise. Teams get the most value from a private evaluation set built from their own real tasks and failures, run repeatedly and tracked over time.
戰略影響
成本與預算
多年來，架構決策決定著效能和營運成本。
更明確的決策
技術教育幫助團隊選擇正確的堆疊，而不僅僅是最新的堆疊。
品質管控
更好的工程選擇可以減少生產中的可靠性事故。
The Future of Evaluating AI Agents
As agents take on longer tasks, evaluations are moving toward longer horizons, more realistic environments, and measures of reliability and cost as well as peak capability. Benchmark saturation and contamination will probably keep pushing developers to create new tasks and to rely more on private, domain-specific test sets. Safety and security evaluations, including resistance to prompt injection, are receiving more attention as agents get access to real systems. There is no agreed standard for grading open-ended agent behavior, so combining outcome checks, calibrated model graders and human review is likely to remain common practice.
現實世界的實施
A coding-agent team scores each attempt by whether the repository's hidden tests pass after the agent's patch, as SWE-bench does, instead of judging whether the diff looks right.
A customer-service agent runs against simulated users and a mock booking database. Graders check whether the final database state matches the correct outcome and whether the agent followed refund policy.
An engineering team runs every task five times and reports how often the agent succeeds on all five, which exposes flaky behavior that a single-run success rate would hide.
A company adds a safety suite in which tasks include a planted instruction to delete files or leak data, and it counts an otherwise successful run as failed if the agent obeyed.
風險與防護欄
優化一項基準測試可以隱藏更廣泛的系統弱點。
基礎設施和維護成本常常被低估。
隨著系統變得更加複雜，安全性和可觀察性差距可能會擴大。
實施路線圖
在實施之前定義延遲、品質和成本目標。
在實際負載和資料條件下進行基準測試。
儀器監控錯誤、漂移和使用者影響。
在擴展之前準備回滾和事件回應路徑。
不斷探索
Free newsletter
Get the daily AI briefing
Three verified AI stories every weekday morning, written in plain English. Free forever, no ads.
One email each weekday. Unsubscribe in one click. We never sell or share your address.
Test yourself
Take the Evaluating AI Agents quiz
Instant feedback on every answer, and a shareable certificate with a verifiable ID once you pass a course.
Support free AI education. AI Understanding is a 501(c)(3) nonprofit — no ads, no paywall, ever. Make a donation
常見問題
What is Evaluating AI Agents?
Evaluating AI agents means measuring how well a multi-step system reaches goals in realistic settings. That covers whether it finished the task, whether the path it took was sensible, whether it used tools correctly, what it cost, and whether it stayed safe. Agents act over many steps and have side effects, so checking a single answer against a reference is not enough, and good evaluations usually run the agent inside a controlled environment and inspect the final state.
Why is checking a single final answer usually not enough for agents?
Multi-step actions change environments, so you need to check what actually happened and how, not just the final text.
How does SWE-bench judge whether an agent solved an issue?
Environment-based checks, here the project's tests, judge whether the fix really works.
What does a pass^k style metric measure?
Requiring success on all k attempts measures consistency, which matters for agents deployed to real users.
Which dimension would flag an agent that succeeds but takes fifty steps for a five-step job?
Trajectory checks judge the path taken, catching loops and waste that outcome checks alone miss.
What is a known weakness of model-based graders?
Model graders are flexible but should be checked against human judgment and watched for systematic bias.
繼續學習
相關指南
為此主題精選的更多指南