When to open this: You need to build "quality guardrails" for an AI Agent product (Coding Agent / customer service chatbot / Research assistant / computer use agent), but you don't know where to start — should you first collect 20 reproducible user cases? Pick an open-source benchmark to run? Start by writing a prompt scoring card? This wiki compresses the methodology that Anthropic's engineering team used internally and with customers in 2026 into a single roadmap.
Core idea in one sentence:"An agent without evals is flying blind in production — you change one part and have no idea what broke, you upgrade the model and don't know if it still works, and when users complain 'it got worse recently' you can only guess. Evals are not a one-time checkbox to pass before launch; they exist to turn 'it got worse' into an engineering problem that is quantifiable, regression-testable, and optimizable."
Who should read this: PMs, engineers, and evaluation leads building AI Agent products. This wiki assumes you have heard of terms like prompt, function calling, and RAG, but does not assume you know SWE-Bench, pass@k, or LLM-as-judge.
0. Why are agents harder to evaluate than LLMs?
The classic form of LLM evaluation: give a prompt, the model outputs a response, and a grader assigns a score.Single-turn, fixed input, fixed output. This is called single-turn evaluation(single-turn evaluation), and it was sufficient in the early LLM era.
Agents are a different beast. They will:
- Talk to themselves over multiple turns (multi-turn)
- Call tools to modify the environment (tool calls modify state)
- Let intermediate results feed back into next-step decisions (adaptive)
These three things mean an agent behaves differently every single run. If you run the same task 100 times, it might succeed 75 times and fail 25 times, and the failure modes are all different.
5 direct consequences:
| Consequence | Implication |
|---|---|
| Errors propagate | One wrong step, and subsequent steps keep reasoning from the incorrect state, leading to total failure at the end |
| Creative workarounds | An agent may find solutions the designers never anticipated — for example, Opus 4.5 exploited a policy loophole on the flight booking task in 𝜏2-bench, which counts as a "failure" under the literal eval rules, but actually delivered a better outcome for the user.A literal fail in eval does not equal a real failure |
| Trajectory variation | The 75 successful runs out of 100 all took different paths. Evaluating the "process" and evaluating the "outcome" are two separate things |
| Environmental noise | Shared state, caches, git history, CPU and memory fluctuations — these factors unrelated to agent capability can contaminate scores |
| Non-deterministic results | With the same prompt and same environment, 10 runs might result in 8 passes and 2 failures.Don't take the number from a single run at face value |
"agent capabilities that make them useful—autonomy, intelligence, flexibility—also make them harder to evaluate"(The three things that make agents useful — autonomy, intelligence, flexibility — are exactly what make them hard to evaluate).
1. Concept Glossary (Chinese-English bilingual, 8 core terms)
You need to align on these 8 terms before writing or reading agent evals. Mixing them up leads to arguments.
| English term | My translation | One-sentence definition | Analogy |
|---|---|---|---|
| evaluation / eval | Evaluation | Provide input + run the model + assign a score = 1 evaluation | A unit test, but for an AI |
| task / problem / test case | Task / Test case | One individual test entry, with an input and success conditions | One exam question |
| trial | Trial / One run | Running the same task one time. Because model outputs are stochastic, you need to run multiple trials | Having one student take the same exam 10 times |
| grader | Grader / Scorer | A piece of scoring logic, which can be code, a model, or a human | The teacher grading the paper |
| assertion / check | Assertion / Check item | A specific judgment inside a grader | One scoring point on an exam question |
| transcript / trace / trajectory | Trajectory / Full record | All outputs from one trial — what the model said, what tools it called, intermediate reasoning | Surveillance footage that records the entire process |
| outcome | Outcome state | What is actually left in the environment when the trial ends. For example, the agent says "booking complete", but the outcome is whether that order record exists in the database | What the agent says and what it actually does are sometimes different |
| harness | Framework / Scaffold | Infrastructure for running evals (eval harness), or infrastructure for making a model run as an agent (agent harness) | The exam room: the full set of mechanisms for proctoring, timing, and collecting papers |
| evaluation suite | Evaluation set | A collection of related tasks (for example, 50 tasks for a customer service scenario) | An exam paper |
Key clarification: agent harness vs eval harness
- agent harness(also called scaffold agent harness scaffold): enables a model to work like an agent — receive inputs, orchestrate tool calls, return results.Claude Code[Anthropic's Agent SDK] itself is an agent harness, exposing primitives via the Agent SDK.
- eval harness: Infrastructure for running evaluations — loading tasks, running trials concurrently, recording transcripts, calling graders, aggregating scores.
"When evaluating an agent, you are evaluating the combination of agent harness + model, not the model in isolation". If you change the harness without changing the model, the score will still change.
2. Three types of graders: Code / Model / Human (how to use them in combination)
The soul of an eval is grader design. Anthropic's practical answer:All three types of graders have shortcomings; use them in combination.
2.1 Code-based grader
Directly use code to judge whether a result is correct.Cheapest, fastest, most reproducible, but brittle.
| What it can do | What it cannot do |
|---|---|
| String match (exact, regex, fuzzy matching) | Judge whether the tone is friendly |
| Binary tests (previously failing cases pass after changes / previously passing cases do not break after changes) | Judge whether an answer is complete |
| Static analysis (lint / type check / security scan) | Tolerate reasonable alternative phrasings or implementations |
| Outcome verification (check whether a record exists in the database) | Judge abstract qualities |
| Tool call verification (which tools were used, whether parameters were correct) | |
| Transcript analysis (how many turns were used, how many tokens were consumed) |
Scenarios to use it: Coding agents (code runs / tests pass = correct), order/refund-type conversational agents (database state is verifiable), any scenario where "right or wrong has an objective standard".
2.2 Model-based grader (LLM-as-judge)
Let another LLM act as the judge, and score according to a scoring rubric.Most flexible, extensible, able to capture subtle differences, but requires calibration, is uncertain, and is expensive.
| Method | Meaning |
|---|---|
| Rubric-based scoring | Give the LLM a scoring sheet, and score item by item |
| Natural language assertions | "Did the agent demonstrate empathy" / "Did the answer cite search results" |
| Pairwise comparison | Which is better, answer A or answer B |
| Reference-based | Compare against a reference answer |
| Multi-judge consensus | Multiple LLMs evaluate, then vote |
Scenarios to use it: Conversational agents (tone / empathy / clarity), Research agents (comprehensiveness of literature reviews), all scenarios requiring "subjective quality" dimensions.
Key discipline: LLM-as-judge must be calibrated against humans. Anthropic's approach:First have both humans and the LLM score the same batch of trials, check the agreement rate; for disagreements, review details and iterate on the rubric. After the agreement rate is sufficient, humans only do sampling checks, and the LLM does the bulk of the work.
"To avoid hallucinations, give the LLM a way out, like providing an instruction to return 'Unknown' when it doesn't have enough information"(To prevent LLM hallucinated scoring: give it an out — allow it to say "I don't know" when it is uncertain).
Don't use one LLM to score N dimensions — split into N independent LLM judges to score separately, which is actually more accurate (if N dimensions are scored together, the LLM will let judgments on one dimension "cross-influence" others).
2.3 Human grader
Human evaluation.Gold standard but slow and expensive.
| Method | Meaning |
|---|---|
| SME review (subject matter expert review) | For domains requiring professional judgment such as legal / medical / financial |
| Crowdsourced scoring | Large-scale subjective quality evaluation (taste, readability) |
| Sampled spot-check | Randomly sample a few entries for human review to calibrate the LLM grader |
| A/B testing | Split real traffic to run two versions, observe user behavior |
| Inter-annotator agreement | Multiple people score the same entry, check the disagreement rate (high disagreement means the task definition is unclear) |
Scenarios to use it:As the gold standard for calibrating LLM graders. Not used daily, but used at key checkpoints — for example, when adding a new category of tasks, or before launching a new model.
2.4 How to score a task as a whole?
A task usually has multiple graders, and there are 3 ways to combine scores:
| Combination method | Meaning | Used for |
|---|---|---|
| Weighted | Each grader gives a score, weighted sum ≥ threshold = pass | Multiple dimensions are all important but partial point loss is allowed |
| Binary | All graders pass = the whole task passes (any fail = fail) | Safety / compliance categories where no deficiency is allowed |
| Mixed | Critical graders use binary, others use weighted | Most practical implementations |
3. Capability evaluation vs Regression evaluation (two tracks, different goals)
The most common mistake: mixing "measuring capability" and "measuring that things don't regress" into one suite.These are two different things, and combining them causes mutual interference.
| Capability eval | Regression eval | |
|---|---|---|
| Chinese term | Capability evaluation / Quality evaluation | Regression evaluation |
| Question it asks | "Can this agent do this X?" | "After changes,can it still do what it used to be able to do?" |
| Expected pass rate | Low(starting at 20-40%) | Close to 100% |
| Purpose | Give the team a "mountain" to climb, show where the agent's capability ceiling is | Prevent "pressing one gourd only to have another pop up" — fixing A breaks B |
| When to run | Model upgrades / major prompt changes | Every PR / every deploy |
Graduation mechanism: After a task in the capability suite runs to near 100% pass rate,it graduates into the regression suite — shifting from "can it do this" to "does it still do this stably".
Why two tracks are mandatory: capability suites need a low starting point to have "room to climb"; regression suites need a 100% starting point to give a signal when things "fall off".If you mix the metrics, you get a useless 60% number that reflects neither capability nor regression.
4. How to evaluate each of the 4 types of Agents (different playbooks)
Agents roughly fall into 4 categories, and their evaluation playbooks differ greatly.Don't copy one playbook to evaluate all agents.
4.1 Coding Agent (code assistant)
Characteristics: Writes code, runs tests, debugs, navigates codebases, runs commands, and has a workflow close to human engineers.
How to evaluate:
- Outcome first: Code runs + all tests pass = the primary standard. SWE-bench Verified and Terminal-Bench both follow this approach.
- For a GitHub issue, the grade is based on "whether the test suite can run successfully without breaking existing tests."
- LLM performance on this benchmark rose from 40% to over 80% in one year.
- Beyond outcome you can also grade the transcript: use heuristic rules to assess code quality, and use an LLM rubric to evaluate how the agent "calls tools / interacts".
Practical example(Evaluating a task of fixing an authentication vulnerability, key scoring dimensions):
deterministic_tests: Deterministic tests must pass (test_empty_pw_rejected.py, etc.)llm_rubric: LLM scores according to the code quality rubricstatic_analysis: Run ruff / mypy / banditstate_check: State check (whether there is an auth_blocked event in the logs)tool_calls: Must have used read_file / edit_file / run_tests- Tracking metrics: n_turns / n_toolcalls / n_total_tokens / time_to_first_token
Discipline:In practice, usually only unit tests + one LLM rubric for code quality are used, other graders are added as needed.Don't stuff 8 graders just for the sake of "completeness", that will turn the eval itself into a source of noise.
4.2 Conversational Agent
Characteristics: Interacts with humans across multiple turns, in domains such as customer service / sales / coaching. It differs from traditional chatbots:It is stateful, calls tools, and takes actions mid-conversation.
How to evaluate:Conversation quality itself is also an evaluation target — not only looking at "whether the result is correct", but also "how the process interaction goes".
Typical evaluation(τ-Bench / τ2-Bench):Use a second LLM to simulate the user, in retail customer service / airline booking scenarios, the agent converses with the "simulated user", and is scored across multiple dimensions after the full process ends.
3 key dimensions:
- state check(State check): Is the ticket status resolved? Is the refund processed?
- transcript constraint(Process constraint): Was it completed within 10 turns?
- LLM rubric(Quality score): Is the tone appropriate? Did it cite policies returned by tools?
Practical example(Evaluating a customer service task of processing a refund):
llm_rubric(Empathy / clear explanation / responses grounded in fetch_policy)state_check(ticket=resolved + refund=processed)tool_calls(Must use verify_identity + process_refund + send_confirmation, and refund amount ≤ 100)transcript(max_turns=10)
Discipline: Conversational agents often have multiple "correct answers" — use LLM-judge more than hard rules. "Rules like 'the agent must call tools in order A→B→C' are too rigid",and tend to penalize reasonable new paths found by the agent.
4.3 Research Agent (Research / Investigation Assistant)
Characteristics: Gathers materials, conducts synthesis and analysis, and produces answers or reports.
Core difficulty:"Is the synthesis comprehensive enough / are the sources authoritative enough / are the conclusions correct" are all relative. A market scan, an M&A due diligence report, and a scientific review each have different standards. Experts also often disagree on what counts as "good enough".
How to evaluate(Combine three types of graders):
- Groundedness check(Groundedness check): Can every conclusion of the agent be traced back to corresponding content in the retrieved sources?
- Coverage check(Coverage check): How many items on the preset list of "key facts that must be mentioned" did the agent mention?
- Source quality check(Source quality): Are the sources used authoritative (rather than just the first search result)?
- Exact match(Exact match): Use exact match for questions with objective answers (such as "What was Company X's Q3 revenue?").
- LLM-judge: Coherence and completeness of open-ended synthesis.
Typical benchmark:BrowseComp — Needle-in-a-haystack on the open web (questions that require complex search + reasoning to answer, but have definite, verifiable answers).
Discipline:The LLM rubric for research agents must be frequently calibrated against humans. Because "how good the synthesis is" is the most subjective, LLM graders are prone to running off with their own hallucinations.
4.4 Computer Use Agent
Characteristics: Interacts with software using screenshots + mouse + keyboard,the same way a human does(rather than calling APIs). It can operate any GUI application, from design tools to legacy enterprise software.
How to evaluate: Must run in a real environment or sandbox environment, then check "whether the agent achieved the goal".
Typical benchmarks:
WebArena: Browser tasks, checked via URL + page state + backend data state (confirm the order was actually placed, not just that the agent reached the "order confirmation page")OSWorld: OS-level tasks, check filesystem state, application config, database contents, UI element attributes after the run
Special tradeoffs for Browser agents: DOM interaction is fast + burns more tokens; screenshot interaction is slow + saves tokens.Anthropic specifically evaluated "whether the agent selected the right tool" in Claude for Chrome — when having Claude summarize Wikipedia, using DOM to extract text is fast; when looking for a laptop case on Amazon, using screenshots is better (because the entire DOM is too heavy).
5. Non-deterministic Math: pass@k vs pass^k
The agent produces different results every run.The "score" for a task is actually a probability distribution, not a deterministic value. Understanding this relies on two metrics.
| Metrics | How to read | Intuition |
|---|---|---|
| pass@k | "Probability of "at least 1 success in k attempts" | Run k times, and you win as long as 1 run passes.The larger k is, the higher the number |
| pass^k | "Probability of "all k attempts succeed" | Every run must pass; the more runs you do, the harder it gets.The larger k is, the lower the number |
Intuitive math(Assuming single-run success rate = 75%):
| k | pass@k (at least 1 pass) | pass^k (all pass) |
|---|---|---|
| 1 | 75% | 75% |
| 3 | 1 − (0.25)³ ≈ 98% | (0.75)³ ≈ 42% |
| 5 | 1 − (0.25)⁵ ≈ 99.9% | (0.75)⁵ ≈ 24% |
| 10 | ≈ 100% | (0.75)¹⁰ ≈ 5.6% |
When k=1 the two are equal(both equal the single-run success rate).When k=10 they tell completely opposite stories — pass@k is close to 100% (as long as you retry enough times, you will eventually hit the right answer), pass^k is close to 0% (it is very hard to be correct every single time consistently).
How to choose:
| Your product form | Choose |
|---|---|
| One success is enough (coding agent submitting a PR, research agent finding an answer, runs in the background, user sees the final result) | pass@k(retries allowed as long as it doesn't frustrate the user to death) |
| User sees every attempt (customer service agent, consumer-grade products — must be correct every time) | pass^k(Consistency is the core; one failure and the user complains) |
Real incident: When Anthropic internally evaluated Claude Code,coding tasks usually cared about pass@1(solved on the first try);consumer-facing conversational agents cared about pass^k(users expect it to be correct every time).
6. An 8-Step Roadmap from 0 to 1
The practical path from "no evals" to "trustworthy evals" given by Anthropic's engineering team.
Step 0: Don't wait for a "perfect dataset", start with 20–50 items
"Teams delay building evals because they think they need hundreds of tasks. In reality, 20-50 simple tasks drawn from real failures is a great start."
Early on, a single change causes large performance differences, so small samples are enough to detect regressions. Only mature agents need large samples to catch small fluctuations.80/20 priority.
Anti-pattern:Waiting until half a year after launch to do evals — by then you are already reverse-engineering "how users use this".
Step 1: Start from "what you manually test"
Cases that devs manually verify before release + bug tracker / customer service tickets.Convert user-reported failure cases into test cases — that naturally gives you priority (user pain points = eval priority).
Step 2: Unambiguous task definitions, with reference solutions
A good task = two domain experts independently looking at the same task can give the same pass / fail judgment. Can they? If not, refine it further.
Anti-pattern(Real Terminal-Bench incident): a task required the agent to write a script, but did not specify the file path. The test assumed a specific path, and if the agent wrote it elsewhere it was counted as fail — that is not the agent's fault, the task is unclear.
Signal:0% pass@100 = the task most likely has a problem, not that the agent is bad. If frontier models fail repeatedly across runs, it is usually because the task description is unclear / the grader has a bug.
Equip every task with a reference solution: an output known to pass all graders.Proves the task is solvable + verifies the grader is correctly configured.
Step 3: Test in both positive and negative directions (avoid one-sided optimization)
If you only test "whether it searches when it should search", you will end up with an agent that "searches everything". You must also test search when it should search + don't search when it shouldn't search.
Real Anthropic Claude.ai incident: When the web search feature launched,testing both sides took many rounds of iteration — both preserving the ability to "search for the weather" and avoiding searching for known answers like "who founded Apple".
"Class-imbalanced evals create one-sided optimization"(Evals with class imbalance breed one-sided optimization).
Step 4: Stable environment + isolated trials
Every trial starts from a clean environment. Shared state (residual files, caches, resource exhaustion) causes correlated failures(multiple trials fail due to the same infrastructure issue, but the data makes it look like the agent is at fault).
Internal Anthropic failure case:Claude cheated by checking git history to get "traces from previous trials". If you don't isolate the environment, the agent can use information left by previous trials to boost its score, which is not real capability.
Step 5: Grader design — don't rigidly enforce execution paths
Common pitfall: Testing "whether the agent called tools A→B→C in order".Too rigid. Agents often find reasonable paths the designers did not think of, and penalizing them is false punishment.
Principle: grade outcomes, not paths(unless the path itself is a compliance requirement).
Partial credit: For composite tasks, an agent that gets half right is clearly better than one that fails immediately.Eval scores should reflect this continuous spectrum, don't do a brute 0/1 cut.
3 commandments of LLM rubric design:
- Give the LLM an "Unknown" exit: allow it to say it doesn't know when it's uncertain, don't force it to give a score
- Structured rubric, score each dimension separately: don't have one LLM evaluate 5 dimensions at the same time
- Frequent human calibration: run batches to check agreement rate between humans and LLM, iterate the rubric on disagreements
Real incident:Opus 4.5 initially only scored 42% on CORE-Bench. Anthropic researchers dug in and found:the grader was too strict(expected "96.124991...", failed the agent for writing "96.12"),task descriptions were ambiguous, and some tasks themselves had randomness and could not be precisely reproduced at all. After fixing bugs + switching to a more lenient grader/scaffold,the score jumped to 95%.
Another case:When METR evaluated Claude, some tasks asked the agent to "optimize to a certain target score", but the grader actually required "exceeding that score". As a result Claude, which followed instructions, was penalized, while those that ignored instructions and charged straight for the score got high marks — that is a task design problem, not the agent being bad.
Step 6: Look at transcripts (required course)
"You won't know if your graders are working well unless you read the transcripts and grades from many trials."
Failures should look fair: looking at the transcript you should be able to clearly state "where the agent went wrong and why". If you can't tell, it might be:
- the agent is truly wrong (fix the agent)
- Unfair grader (fix the grader)
- Unclear task (fix the task)
Anthropic internally invests in dedicated tools to review transcripts. Reviewing transcripts is a core skill for agent development, not an optional extra.
Step 7: Monitor "eval saturation"
Eval saturation(Eval saturation): When the agent passes all solvable tasks, there is no more room for improvement on the eval. A 100% score yields no signal.
Signal: After SWE-Bench Verified rose from 30% to over 80%,the remaining 20% are all the hardest cases, so progress appears to slow down. At this point improvements may be real (harder tasks are being solved), but they show up as only small score gains.
Correspondingly:Qodo(a code review startup) initially used one-shot eval to assess Opus 4.5 and thought there was no improvement; only after switching to an agentic eval framework (allowing multi-turn interaction and tool use) did they see the real improvement.If your eval cannot keep up with model upgrades, you will draw wrong conclusions.
Iron rule:Do not use eval scores directly — someone must dig into the details and review transcripts before drawing conclusions.
Step 8: Open contributions + continuous maintenance
An eval suite is a living artifact(living artifact), not a one-time check done right before release.
Anthropic internal practice:
- A dedicated eval team maintains the core infrastructure
- Domain experts + product teams contribute eval tasks — whoever is closest to users knows best what "success" looks like
- PMs / Customer Success / Sales can all use Claude Code to turn "success criteria" into eval tasks and submit PRs
Golden quote:"For AI product teams, maintaining evals should be treated as day-to-day work just like maintaining unit tests".
Eval-driven development(Eval-driven development):First write evals to define "what the agent should be able to do in the future", then iterate on the agent. Today it can do 70%; bet that in 6 months model capabilities will rise to 95%. As soon as a new model comes out, running the suite tells you whether the bet paid off.
7. Evals do not work alone — use 5 methods together (Swiss Cheese Model)
Relying only on automated evals causes misses. Anthropic advocates the Swiss Cheese Model(Swiss Cheese Model, originating from safety engineering):Every layer has holes, but combined they block most holes.
| Method | Strengths | Weaknesses | When to use |
|---|---|---|---|
| Automated evals (Main battlefield of this wiki) | Fast / reproducible / does not disturb users / runs on every commit | High upfront investment / prone to drift from real usage | Before launch + CI/CD(First line of quality defense) |
| Production monitoring(Online monitoring) | Real user behavior / catches what evals miss | Reactive (users have already hit the problem) / noisy signal / no clear "right or wrong" | After launch(Detect distribution drift + anomalies) |
| A/B testing | Measures real user outcomes (retention / completion rate) / controls confounders | Slow (must wait for statistical significance) / can only test deployed versions / unclear "why it went up" | Before launch for major changes |
| User feedback(thumbs down / bug reports) | Discovers failure modes you did not think of / comes from real users | Sparse / selection bias (only frustrated users give feedback) / users do not explain "why" | Do continuously, triage regularly |
| Manual transcript review | Catches subtle issues missed by automated checks / trains intuition | Slow / not scalable / rater fatigue | Weekly sampling + deep dives as needed |
| Systematic human research | Gold standard / handles subjectivity / calibrates LLM graders | Expensive / slow / inter-rater disagreements require coordination | Key milestones (new model / new task category) |
"No single evaluation layer catches every issue. With multiple methods combined, failures that slip through one layer are caught by another."
Translation: No single layer of evaluation can catch every problem. Combine multiple methods so that what one layer misses, another catches.
8. Eval framework ecosystem (2026)
You do not have to write a harness from scratch.Pick one mainstream open-source or commercial framework to use, and spend your effort on tasks / graders.
| Framework | One-liner | Best for |
|---|---|---|
| Harbor | Containerized agent execution + large-scale cross-cloud concurrency + standardized task / grader formats | Coding agents / Terminal-Bench 2.0 and similar use cases requiring heavy infrastructure |
| Braintrust | One-stop shop for offline eval + online observability + experiment tracking | Teams that need both development iteration and online monitoring |
| LangSmith | Trace + offline/online eval + dataset management, LangChain ecosystem | Teams already using LangChain |
| Langfuse | Similar in functionality to LangSmith,open-source and self-hostable | Teams with strict data compliance requirements |
| Arize Phoenix / AX | LLM trace + debug + eval, Phoenix open-source / AX commercial | Mid-to-large production deployments |
Discipline:"Frameworks are just accelerators; what really determines the value of an eval is the tasks and graders you write". Quickly pick a framework that fits your workflow first, and invest your effort in the content — don't spend a month comparing frameworks.
Many teams start by combining multiple tools or cobbling together simple scripts, and only later adopt heavy frameworks.
9. Anti-pattern checklist (9 real-world failure cases)
| # | Mistake | Real cost |
|---|---|---|
| 1 | Wait for a "perfect dataset" before doing evals | You only do evals half a year after launch, and can only reverse-engineer success criteria |
| 2 | Use one eval suite to both measure capability and guard against regressions | A starting score of 60% measures neither capability nor regression well |
| 3 | Rigidly require the agent to call tools in A→B→C order | The agent is penalized for finding a reasonable new path |
| 4 | Have one LLM grade 5 dimensions at the same time | Dimensions interfere with each other, making results inaccurate |
| 5 | Do not give the LLM-judge an "Unknown" option | Forcing the LLM to give a score = manufacturing hallucinations |
| 6 | Share environments across trials / no isolation | In Anthropic's internal evals, Claude"cheated" using git history(real example) |
| 7 | Tasks do not specify concrete paths / key parameters | Real Terminal-Bench incident: the agent was not wrong; the task was unclear |
| 8 | Draw conclusions directly from eval scores without digging into details | Opus 4.5 appeared to score 42% on CORE-Bench; deep investigation revealed a grader bug, and the actual score was 95% |
| 9 | Imbalanced task categories (only positive examples / only negative examples) | Anthropic Claude.ai web search initially optimized one-sidedly, and only calibrated after repeated iterations |
10. Self-check checklist (for product / engineering teams)
Go through this table,start at 0 points, 1 point per item, full score 12:
□ 我的 agent 是哪一类(coding / 对话 / research / computer-use)?
□ 我的 capability suite 和 regression suite 是分开的吗?
□ 至少有 20 个 task?其中至少有 5 个是"真实用户失败"转来的?
□ 每个 task 有参考解,且参考解能过所有 grader?
□ 至少 3 类 grader 至少有 2 类在用(代码 / 模型 / 人工)?
□ LLM-judge 跟人工校准过一致率?
□ 评的是 outcome,不是硬卡 tool calls 顺序?
□ trial 之间环境隔离(干净起步,不共享 state)?
□ 我看过最近 10 条 failure 的 transcript 吗?
□ 我的 capability eval 起点是 20-40%(有爬山空间),不是 80%(快饱和)?
□ pass@k 还是 pass^k 用对了(看产品形态)?
□ eval task 是开放贡献的,产品团队 + 客服都能加 task?
< 6 points: Still in the "flying blind online" stage; first build a capability suite and review a few transcripts 6-9 points: Basics are OK; add a regression suite and calibrate LLM-judges 10-12 points: Entered eval-driven development; you can bet on capability improvements over the next 6 months
11. Related wiki / cross-reference map
| If you want to read | wiki |
|---|---|
| Agent architecture layer (6 perspectives: Workflow vs Agent / Multi-Agent orchestration / Sub-Agent philosophy / framework comparison) | AI Agent Architecture and Multi-Agent Orchestration (cross-author topic synthesis) |
| Claude Code workflow (6 perspectives: engineering division of labor / Skills / MCP / terminal window as Agent) | Claude Code Workflow (cross-author topic synthesis) |
| Claude Skill engineering (Skill = SKILL.md + references/ + scripts/ three-piece set) | Claude Skill Engineering (cross-author topic synthesis) |
| Other writings from the Anthropic engineering team (coding agent multi-harness design / agent skills / MCP / context engineering) | (This batch of Phase B outputs Anthropic engineering team first-party perspective: Agent engineering / Skills / MCP / Claude Code (2025-2026 complete collection)) |
| Wang Kai's hands-on Agent perspective (Agent leveling + 6-model experiments + terminal window as Agent) | Wang Kai's Hands-on Agent Engineering |
| Lao Huang's post-level judgments on Agent architecture (Workflow vs Agent + Multi-Agent critical points + 3 delivery-rate rules) | newtype · Agent Architecture and Multi-Agent Orchestration |
| Lenny / Cat Wu (Anthropic Head of Product for Claude Code) interview perspective | Lenny's Newsletter / 2026 Three-Person Talk on Organizational Operations in the AI Era (Anthropic + OpenAI) |
12. Meta information
Source:sources/reference/anthropic/engineering/demystifying-evals-for-ai-agents/index.md(Published by the Anthropic engineering team in 2026, authors Mikaela Grace / Jeremy Hadfield / Rodrigo Olivares / Jiri De Jonghe; original URL https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents).
Rewriting strategy: Topic-driven rewriting — broken into 12 sections, reorganized from a PM perspective (starting from "why agents are harder to evaluate than LLMs", rather than the original "introduction → structure" order); added PM-friendly Chinese explanations, life analogies, and a self-check checklist; key technical details (grader matrix, Step 0-8, pass@k math) are faithfully preserved. No more than 3 sentences are quoted directly from the original, and long quotes are accompanied by Chinese translation.
Bilingual strategy(translation_status: bilingual): Core English terms (eval / task / trial / grader / transcript / outcome / harness) are given Chinese translations on first appearance + Chinese-English comparison in tables; golden quotes from Anthropic's original text are kept in English + *译:中文*; section titles mix Chinese and English (product names / benchmark names / term abbreviations are not translated).
Phase A purpose: Serves as the density / layout / bilingual style baseline for this anthropic-* batch. Subsequent Phase B-D wikis will follow this template.
Sources and related materials
- https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents
- AI Agent Architecture and Multi-Agent Orchestration (cross-author topic synthesis)
- Claude Code Workflow (cross-author topic synthesis)
- Claude Skill Engineering (cross-author topic synthesis)
- Wang Kai's Hands-on Agent Engineering
- newtype · Agent Architecture and Multi-Agent Orchestration