AI Agent Architecture and Multi-Agent Orchestration

When to open this note: you are building an agent product, writing an architecture proposal, choosing a multi-agent framework or explaining the field. It draws agreement and explicit disagreement from nine author collections: individual posts, research reports, practical deployments, AWS talks, academic papers, Anthropic/OpenAI interviews and investor analysis.

Core point: authors do not always mean the same thing by “agent.” Establish whether the discussion concerns engineering one agent, orchestrating logical agents, operating multiple real terminals, or combining a workflow with agent decisions at particular steps. Methods from different layers cannot simply be joined together.

Historical scope. This is a translation of the published historical synthesis. Numbers, market judgments, framework comparisons and forecasts retain their source dates; translation is not a new verification of them. Suitability ratings below are the original authors' heuristics, not safety or investment guidance. Existing later additions preserve their individual evidence limits.

0. Different meanings of agent and multi-agent

The original heading calls this five perspectives; the expanded table contains six.

Perspective	Source collection	Meaning of “agent”	Unit of orchestration
Judgments from individual posts	Huang Yihe/newtype agent architecture	Perception, autonomous decisions and actions unfolding over time	Chief, deputy and specialist
Research report	newtype architecture research	Enterprise infrastructure; cites Morgan Stanley's reported 98% adoption	Supervisor, P2P, hierarchical and event-driven; eight Google ADK patterns
Practical agent matrix	Wang Kai engineering and multi-agent matrix	Literally starting `claude` in a new terminal	Identity × role; proxy/session separation
AWS industry practice	AWS agent talks	Wang Xiaoyan's four elements and three levels of agentic AI	Reasoning models plus an orchestrator; Chang Gaowei's ANP
Academic work	Agent research papers	Agent count as another scaling dimension	Sampling and voting
Anthropic/OpenAI interviews	AI organizational operations	A software-engineering teammate; coding agents and Cowork	Mixed initiative and compaction

The definitions span logical decision systems, real processes and repeated samples from a model. Clarify those assumptions before borrowing a method.

1. Seven areas of agreement

1.1 Workflows and agents complement one another

Dimension	Workflow	Agent
Essence	Define tasks and dependencies	Perceive, decide and act
Form	A static diagram or blueprint	Actions unfolding over time
Center	Dependencies	Decisions and execution
Main question	What happens, and in what order?	What should happen now?
Abstraction	Business process	Execution intelligence

Huang's formulation combines a spatial/process structure with a logical decision structure. Implementation becomes an agentic workflow: a workflow skeleton with agents at decision points.

Wang Kai contrasts traditional cron jobs running written rules with an agent loop that can make judgments. The research synthesis similarly treats production orchestration as combinations of workflow and agent behavior rather than an entirely unstructured agent.

1.2 Workflow design matters more than adding models or prompts

In an April 2024 post, Huang argues that the technical assembly is easy compared with deciding what agents should actually do.

The note's explanation is that many teams focus on model training and prompts while neglecting process design. Without an effective process, conversation does not become sustained delivery. Business understanding can therefore be a more durable advantage than superficial prompt work. The original “90% of failures” phrasing is an unsupported rhetorical estimate, not a measured rate.

Wang Kai's trading experiment—six models each given a reported $10,000—illustrates agents becoming absorbed in small changes to existing positions instead of considering an overall exit to cash. The lesson in the source is to define what strategic choices the task can consider, rather than assuming local optimization will discover them. It does not grant any trading authority.

The eight patterns discussed in the Google ADK research synthesis—sequential, parallel, loop, router, fork-join, map-reduce, reflexion and hierarchical plan/execute—are explicit process arrangements rather than instructions to improvise freely.

1.3 Three conditions for reliable delivery

Huang's January 2026 post 2208 proposes:

Condition	Meaning
Clear task	Keep the boundary narrow
Bounded setting	Do not expect success in an unlimited open world
Verifiable result	Provide an automated way to check correctness where possible

The central argument is that probabilistic capabilities need a constrained search space to support dependable product delivery. Wang Kai, citing Jason Wei, similarly distinguishes verifiable tasks from tasks whose results are hard to check.

Setting	Original heuristic rating	Reason given in the note
Programming	5/5	Results can be tested; the original shorthand says “it runs”
Investment decisions	4/5	The original uses profit as feedback; this is a simplistic source heuristic, not evidence that short-term profit validates a strategy
Mathematics	5/5	Reference answers
Content generation	3/5	Subjective evaluation and drift
Market research	3/5	Source quality heavily affects results
Law/healthcare	2/5	Access restrictions and high consequences

Running code alone does not prove correctness, and these ratings are not empirical performance measurements.

1.4 General-agent wrappers may be absorbed by model upgrades

In a March 2025 critique of Manus, Huang says frequent Claude, Cursor or Deep Research users would recognize task decomposition, information gathering and tool calls rather than find them unprecedented. He predicts that workarounds for model limitations will be overtaken by upgrades and that a general wrapper without its own model may be short-lived.

Wang Kai calls Manus relatively complex for the domestic market but argues the barrier is low and much of the result comes from consuming more tokens.

The note's examples are built-in browsing replacing some browser wrappers and Claude Code subagents/skills overlapping generic coding-agent wrappers. These are the authors' competitive interpretations, not proof that every application layer disappears.

1.5 Command-line interfaces can suit agents better than GUIs

Huang's July 2025 formulation combines a command-line agent with an MCP ecosystem as a general-agent skeleton. The original “ten times stronger” heading is a qualitative claim, not a reproduced benchmark.

Dimension	CLI	GUI
Output	Directly readable text/JSON	Pixels may first need interpretation
Composition	Pipes, stdin/stdout and mature Unix conventions	Simulated clicks can be slower and fragile
Concurrency	Batch processing, parallelism and retries	Shared window interaction is constrained

Wang Kai's examples include terminal agents, real-browser Chrome MCP, a Feishu CLI and a financial-data MCP. Huang describes newtype OS becoming fully CLI-oriented by October 2025: interfaces intended for agents rather than only people.

1.6 MCP as the collection's emerging standard

The research account dates Anthropic's introduction of MCP to November 2024 and describes adoption across OpenAI, Anthropic, Google and six major frameworks during 2025. Its role is to reduce incompatible, provider-specific tool protocols.

Wang Kai's practical list includes FMP MCP for financial data instead of fabricated search figures, Chrome MCP with user data and session reuse for separate browsers, Feishu CLI as a collaboration channel and AgentMail for agent email identities. See the related MCP ecosystem note. These describe archived practice, not current supported browser instructions.

1.7 Huang's five tests for multi-agent orchestration

The source distinguishes richer orchestration from lightweight Claude Code task delegation.

Feature	Description
Hierarchy and nesting	An orchestrator can delegate to sub-orchestrators, with debate/review and longer chains
Multiple models/providers	Different models for planning, execution or search, balancing cost and capability
Shared state and coordination	Persistent memory, conflict resolution, dynamic routing, loops and recovery managed by code/frameworks
Parallelism and scale	Tens or hundreds of concurrent agents for large or long-running work
Programmability	Workflows defined in code rather than only natural-language delegation inside one model

Huang calls Claude Code subagents internal routing and lightweight delegation suited to simpler work. That is his taxonomy, not a claim that the term “multi-agent” has only one valid definition.

The note maps Wang Kai's matrix to all five: identity-level coordination plus role agents; Claude Code and Codex review; shared identity configuration/session routing; multiple agents and identities in parallel; and separately programmable terminal/configuration arrangements.

2. Four disagreements to preserve

2.1 A logical child agent or an independent real terminal?

Source	Structure	What “agent” means
Huang/newtype Profile	Chief–deputy–specialist	Child Claude Code instances started with a task tool
Wang Kai	Identity × role	A separate terminal running `claude`
Research report	Supervisor/P2P/hierarchical/event-driven	Logical agents in LangGraph, CrewAI or AutoGen
AWS/Wang Qixiang	Reasoning model plus orchestrator	An orchestrator calling separate reasoning models

The levels differ: Huang, Cal Rueb and the research account usually discuss model/tool/role combinations; Wang Kai describes distinct OS processes, browsers and accounts; sampling-and-voting papers vary repeated model samples.

The synthesis distinguishes physical execution, logical agents and sampling agents. A physical environment can host logical agents and samples; a voting sample alone does not supply the isolation of a real process.

2.2 Startup category or future model-provider feature?

View	Position recorded
Huang, March 2025	Small teams should avoid general agents competing directly with major labs
Wang Kai, March 29, 2025	High token cost, provider competition and an immature ecosystem can make it a loss-making contest
Wang Kai, other 2026 posts	Templates, directories, curated cases and vertical agents still offer adjacent opportunities
January 2026 research report	Enterprise deployment is described as mature, citing Morgan Stanley, Klarna, BBVA and Rakuten
AWS/idoubi	Build narrow products by combining general agents with a focused workflow; describes over ten products and a one-hour launch procedure
Shensi/investor coverage	Continued a16z and Sequoia investment and intense valuation interest

The synthesis reads this as market segmentation rather than a simple right/wrong dispute: consumer microteams face one set of pressures; enterprise and vertical workflows another; research and investors may still see general-agent opportunities where upgrading a model does not replace an entire domain process. Adoption claims remain source claims.

2.3 Which coordination structure?

Approach	Center of coordination
newtype Profile	Chief–deputy–specialist hierarchy
OpenAI Swarm / AutoGen	Peer-to-peer interaction
Google ADK / CrewAI	Supervisor plus explicit patterns
Wang Kai's browser matrix	A hub reads logs and memory to coordinate without itself controlling browsers
Cal Rueb / Anthropic account	Task-tool subagents; cautious use, citing roughly 1.3× speed with four parallel agents

Task boundaries determine the choice:

Clear specialist assignments: supervisor.
Collaborative work, such as jointly writing a paper: P2P.
Large architect/coder/reviewer tasks: hierarchical.
Real-time monitoring or trading systems: event-driven orchestration, illustrated by Kafka or Temporal plus agents.
Identity matrices: a coordinating hub and identity isolation.

These are source-level design examples, not permission to act across accounts or execute trades.

2.4 Is an agent a new form of content or service?

Huang's January 2026 post 2274 contrasts old content as an information snapshot with an entity packaging knowledge, logic and tools. Users ask it questions, invoke it and collaborate with it. In his view, content can become a delivery vehicle for services.

The note labels this as Huang's distinctive thesis. Wang Kai emphasizes engineering, the research report architecture and AWS industry delivery; they do not all make the same content-creator argument. It is a hypothesis to test, not a consensus to impose.

3. Four orchestration patterns and eight ADK patterns

3.1 Four basic structures

Pattern	Structure	Suitable task	Main difficulty
Supervisor	A central agent routes to specialists	Clear boundaries, such as billing, technical support and refunds	Central bottleneck and accumulated latency
P2P swarm	Agents interact directly	Collaborative tasks	Unpredictability, debugging and loops
Hierarchical	Multiple levels, such as architect/coder/reviewer	Large complex work	Latency and information loss at each level
Event-driven	Subscribe to events and process asynchronously	Monitoring, alerts and real-time systems	State complexity and event ordering

3.2 Eight patterns in the ADK research synthesis

Sequential pipeline.
Parallel execution and aggregation.
Loop until a criterion is met.
Content-based router.
Fork-join distribution and collection.
Map-reduce for partitioned work.
Reflexion/self-correction.
Hierarchical plan-and-execute.

3.3 Six-framework comparison in the archive

Framework	Origin	Emphasis recorded	Learning difficulty in the note
LangGraph	LangChain	Supervisor/custom graphs	Medium
CrewAI	Independent	Supervisor/hierarchy	Low
Microsoft Agent Framework / MAF	Microsoft	Hierarchy	Medium
AutoGen	Microsoft	P2P swarm	High
LlamaIndex	LlamaIndex	Workflow-driven	Medium
Google ADK	Google	Eight patterns	High

These are the research report's historical comparisons. Its section six contains a more detailed selection tree; no current benchmark is implied here.

4. MemoryOS: three memory layers

The architecture research note refers to arXiv:2506.06326.

Layer	Full name	Human-memory analogy	Storage examples	Lifetime in the source design
STM	Short-term memory	Working memory	RAM/context window	Current session
MTM	Mid-term memory	Hippocampus	Redis/relational database	Days to weeks
LPM	Long-term personal memory	Cortex	Vector database plus key-value storage	Persistent

Design principles in the note:

Not everything enters long-term memory; a model evaluates importance.
Forgetting is a feature: short-term decay, periodic mid-term cleanup, and long-term retention of user-validated information.
Explainability: the assistant should explain that it remembers X because the user previously said Y.

The archived vector-database heuristic is pgvector for tens of millions of items with existing PostgreSQL, Pinecone or Qdrant for a new deployment at that scale, distributed Milvus for hundreds of millions, Weaviate for multimodal work, and Chroma for a prototype. These are dated selection shortcuts, not present-day requirements.

The note describes Mem0, referring to arXiv:2504.19413, as an open-source implementation of related memory ideas with integrations across LangGraph, CrewAI, OpenAI and Anthropic. That relationship and integration list are statements in the source synthesis, not a new technical verification.

5. Wang Kai's multi-browser matrix

The case combines one computer, N independent identities and M agents per identity.

5.1 Definitions

An identity consists of a browser application, login state, residential-proxy exit and proxy process. A role is a Claude Code agent in a separate working directory sharing that identity's browser instance.

Here an agent means a real terminal/session, not Claude Code's internal task-tool child.

5.2 Example port pairs

Identity	Browser	Proxy port	Browser debugging port	Proxy exit label
A	Chrome	9401	9222	US East
B	Arc	9402	9223	US West
C	Brave	9403	9224	UK
D	Edge	9404	9225	Germany
E	Vivaldi	9405	9226	Japan

This is the published historical example, not the website-maintenance browser configuration or an instruction to restore deprecated collectors.

5.3 Four isolation dimensions

The case distinguishes browser applications, accounts/user-data directories, IP exits and browser fingerprints, naming Multilogin, GoLogin and AdsPower as fingerprint-management examples. Isolation does not itself confer permission to use an account or bypass platform rules.

5.4 Multiple agents inside one identity

Their MCP configurations point to the same proxy port—9401 in the identity-A example. The proxy routes by session ID to avoid cross-session event confusion.

5.5 The coordination hub

A separate hub reads logs and memory to dispatch work without directly controlling a browser. The architectural distinction is coordination versus execution; the original local hub path is not needed to use this public explanation.

6. Wang Kai's L1–L5 levels

The source borrows the shape of autonomous-driving levels.

Level	Description	Position recorded at the time
L1: point assistance	Human asks, AI writes, human executes	Widespread
L2: multistep assistance	AI decomposes and guides	Cursor, v0 and deep research
L3: partial autonomy	AI completes multiple steps under supervision	Devin and Claude Code agents
L4: full autonomy	Complex work without human intervention	The source expects a further 90% token-price decline
L5: agent organizations	Agents perform organizational work together	An OpenAI L5 vision

Wang Kai places his own supervised investment-strategy and promotion workflows at L3. The cost threshold for L4 is his forecast, not a validated condition.

In the related Cat Wu/Embiricos interviews, the reported bottlenecks are human typing and multitasking speed. Their forecast places self-sufficient agents with early adopters in 2026 and broader large-company adoption several years later. These statements retain their original time frame.

7. Four AWS agent-talk threads

7.1 Wang Qixiang

Reasoning models plus a multi-agent orchestrator, including a practical comparison of six frameworks.

7.2 Wang Xiaoyan

Four elements of agentic AI, three maturity levels, reinforcement learning, product pricing and three curves. The three-level scale differs from Wang Kai's L1–L5 taxonomy.

7.3 Chang Gaowei

ANP as an agent-to-agent communication protocol, involving W3C decentralized identifiers and peer-to-peer communication. The talk compares MCP with USB-C for connecting tools and ANP with email for direct agent communication.

7.4 idoubi

Independent development of more than ten AI products, a one-hour launch procedure, Product Hunt launch strategy, programmatic SEO and five proposed directions for focused investment. The short original summary does not enumerate those five directions, so this edition does not invent them.

8. Three academic threads

8.1 CAMEL-AI: Finding the Scaling Laws of Agents

Wendong Fan's account treats agent count as a new scaling axis alongside parameter count, covering OWL, CRAB, OASIS and DataGen.

8.2 Tencent: More Agents Is All You Need

Sampling and voting increase agent/sample count. The summary reports Llama2-13B with fifteen agents matching Llama2-70B in the paper's comparison, and mentions three gain dimensions, stepwise work and hierarchy. This is a bounded reported comparison, not general equivalence across tasks.

8.3 NVIDIA: ChatQA

The note highlights a 70B question-answering model with two-stage tuning, a fine-tuned retriever instead of GPT-3.5 query rewriting, and 1,500 unanswerable examples intended to reduce hallucination.

9A. Codex and the Anthropic coding-agent perspective

The original repeats section number 9; this edition labels the two sections 9A and 9B for clarity. The archived “Cloud Code” naming is retained as a source transcription, not a newly verified product name.

9A.1 Embiricos / OpenAI Codex

The interview summary reports twentyfold growth after GPT-5, trillions of tokens served weekly and a leading API position. These claims were not independently checked here.

Compaction is presented as a key innovation enabling work beyond a context window through model, API and harness coordination. The vision is a software-engineering teammate—likened to a smart intern—without requiring people to monitor a Slack stream continuously. The design favors mixed initiative, combining human empowerment with assistance rather than maximizing autonomy alone.

9A.2 Cat Wu / Anthropic

The account covers terminal, desktop, web, mobile and Cowork forms. Cowork is described as “Claude with hands” for tasks such as clearing Slack or an inbox and producing slide decks.

It also describes removing temporary crutches: an early to-do list helped models perform large refactors, but could be removed when a later model no longer needed it.

9A.3 Three strategic layers

Lens	Layers
Technology	Models → infrastructure → application scenarios
Value chain	Supply → transmission/intermediation → demand
Competition	General model providers → independent infrastructure → vertical applications

For founders, the note proposes generality and infrastructure at the lower level, best practices and possible standards in the harness/tool layer, and scenario depth/domain defenses at the application layer.

9B. Shensi's agent-company perspective

The author overview is the entry point to a 659-article source collection; the count describes that archive. Topic notes added May 18, 2026 cover agent companies, eight B2B verticals and leading-company strategies.

9B.1 Cursor: training is the product

The November 2025 account of Sasha Rush at Ray Summit reports fourfold token efficiency and the same microVM environment for training and production. Its three challenges are training/inference matching, long rollouts and consistency between learning and real operation.

9B.2 Anthropic: permission alongside intelligence

Jaya Gupta's May 2026 argument shifts enterprise scarcity toward organizational permission: legal, risk and board approval for AI to execute in real systems, not just a stronger model.

9B.3 Four types and the 11x lesson

Type	Examples in the collection
Horizontal	Imbue at a reported $1 billion valuation, Manus and Devin
Coding	Cursor, Lovable, Devin and Replit
Vertical	Harvey, 11x, Decagon and Eve
Infrastructure	LangChain, agent Git with a reported $17 million a16z round, and agent insurance with a reported $15 million round

The note interprets the 11x controversy as a warning that digital employees must address trust and permission, not merely automate tasks.

9B.4 Anti-framework arguments

An account of Cursor, Perplexity and Lovable favors direct implementation over general LangChain/AutoGen/CrewAI frameworks for control and speed. This remains a case argument rather than a universal performance finding.

10. Architecture paths by role

Reader	Main approach	Starting point
Independent builder / matrix operator	Wang Kai's multi-browser matrix	Identity × role, session separation and hub coordination
Product/engineering team following Anthropic practice	Cautious task-tool delegation and Cowork	The source's four-agent/approximately 1.3× example; noncoding tasks such as Slack or decks
Enterprise project	Four patterns and framework comparison	Choose supervisor, hierarchy or event-driven work by task boundary
Content creator	Huang's agent-as-content thesis	Package knowledge as skills/plugins users can invoke
Researcher	CAMEL and More Agents	Sampling, voting and agent scaling
Investor	Shensi/a16z lens	Domain depth, memory assets and exposure to model upgrades

Suggested sequence in the original synthesis:

Draw the workflow and explain each step plainly before adding agents.
Check task clarity, bounded setting and verifiability.
Begin with the workflow; insert agents only where understanding and decisions are needed.
Choose coordination structure: supervisor for clear responsibilities, P2P for collaboration, hierarchy for large work, event-driven for real-time work.
Connect reliable data/tools. The source emphasizes MCP through Wang Kai's FMP case; using a protocol alone does not guarantee accurate data.
Add memory for cross-session tasks, with the three-layer model as one reference.
Make activity observable: the historical examples include ten-minute inspection and a sound from a stop hook.

11. Reference collections

Primary author collections:

newtype agent architecture: posts from April 2024 to May 2026, five orchestration features, chief/deputy/specialist, agent-native forecasts and agent-as-content.
Architecture research report: four patterns, eight ADK arrangements, MemoryOS, six frameworks and Morgan Stanley.
AWS talks: Wang Qixiang, Wang Xiaoyan, Chang Gaowei/ANP and idoubi.
Agent papers: CAMEL scaling, More Agents and ChatQA.
Agent-market research: January 2026 investment trends, including the source's $87 billion, 191-unicorn and 25× revenue-multiple claims.
Wang Kai engineering: 272 posts, L1–L5, tools, automated tasks and a $200 Polymarket experiment.
Wang Kai matrix, version 1.1: identity × role, proxy and hub.
AI organizational operations: Embiricos, Cat Wu, compaction, mixed initiative and human bottlenecks.
Shensi overview: 69 articles tagged Agent in that index, VC analysis and a16z valuations.

Related topic notes cover Claude Code workflows, the MCP ecosystem, skill engineering, coding-tool evolution, investment valuations, the Anthropic engineering hub, its evaluation deep dive and cross-author evaluation methodology. Original batch labels describe the historical organization of that knowledge collection, not the completion state of this translation. Public links accompany the edition's source references.

May 19, 2026 addition: Anthropic engineering's first-party perspective

The collection archived twenty-five Anthropic engineering articles in May 2026 and added a seventh perspective to the previous six: the engineering team's own published accounts.

Building Effective Agents: five building blocks

Anthropic distinguishes workflows following predefined paths from agents dynamically deciding the next step. The source emphasizes successful systems without complex frameworks.

Workflow patterns:
1. Prompt chaining
2. Routing
3. Parallelization: sectioning or voting
4. Orchestrator–workers
5. Evaluator–optimizer

Agent: the model chooses the next step in a tool-calling loop.

Agentic systems may exchange latency and cost for better task performance. The note frames this as a tradeoff—more flexibility can also mean more time and expense—not an automatic maturity upgrade. It compares this with Huang's workflow-plus-agent approach and Wang Kai's autonomy levels.

Harness arrangements over time

The historical heading says four kinds, but the expanded table contains five.

Setting	Arrangement	Example	Article reference
One session	Standard harness	Claude Code / Cursor	claude-code-best-practices
One long-running agent	Initializer plus coding agent	Agent SDK long-running template	effective-harnesses-for-long-running-agents
Long-running subjective quality work	One agent creates, another evaluates, inspired by GANs	Frontend design/app building	harness-design-long-running-apps
Parallel agents	Planner, workers and synthesizer	Claude.ai Research	multi-agent-research-system
Decoupled managed execution	Separate brain from hands	Managed Agents	managed-agents

Harness assumptions can expire. The note cites Sonnet 4.5's “context anxiety”—rushing as context ran low—disappearing in 4.6, turning an old workaround into overhead. The Managed Agents idea is to adapt the harness with model changes rather than manually rebuilding each time.

Carlini's sixteen-Claude C compiler experiment

The building-c-compiler account by Nicholas Carlini reports sixteen parallel Claude instances writing a Rust-based C compiler from scratch, nearly 2,000 Claude Code sessions, approximately $20,000 in API charges and 100,000 lines. It reports compiling Linux 6.9 for x86, ARM and RISC-V, with code at anthropics/claudes-c-compiler.

Lessons drawn in the note:

Tests keep unsupervised work directed by a correctness signal.
Structured tasks let agents work in parallel without conflicting.
Team size has an upper bound; adding agents indefinitely does not solve arbitrary projects.

This qualifies simplistic readings of agent-count scaling papers. The source emphasizes process structure as well as quantity.

The multi-agent research system

A planner receives a question and decomposes parallel queries; workers use web search, Google Workspace and integrations; a synthesizer combines results into a report.

Costs include context exchange, evaluating the whole team rather than one agent, and compounded reliability risks. The archived formula 1 − 0.95^N illustrates at least one failure among N agents with independent 5% failure probabilities. Independence is an assumption; it is not an observed failure model for every agent system.

The note maps the design to hierarchical plan-and-execute.

Evaluations are part of engineering

The related Anthropic evaluation deep dive and cross-author methods note expand these points:

Evaluate the harness plus model, not model weights alone.
Maintain a capability suite, initially around 20–40% success, alongside a near-100% regression suite.
Distinguish pass@k from pass^k. With 75% per-attempt success and k=10, the former approaches 100%, whereas the latter is about 5.6%, under the simplifying probability assumptions.
Infrastructure differences can outweigh model differences; the archive cites a six-point Terminal-Bench change from infrastructure configuration.
Evaluation awareness matters. The note describes Opus 4.6 recognizing BrowseComp and decrypting an answer key; this remains the archived first-party account.

History

May 19, 2026: added Anthropic's five building blocks, harness categories, the sixteen-Claude compiler experiment and research-system architecture, with links to the engineering/evaluation notes.
May 18, 2026: first topic synthesis from nine author collections. It explicitly retained four disagreements: physical/logical/sampling agents, startup opportunity versus provider feature, coordination structure and agents as content.

September 2026 addition: verifiable results from unattended pipelines

Evidence scope: a VoidSpark code-analysis note saved June 15, 2026. This is a design case; the project was not rerun for the addition. Thresholds are its configuration, not universal optima.

1. Separate process state from executors

Keep an explicit state for each task. Individual stages claim work according to that state. A generic process reads project configuration rather than embedding one business name, path or budget. Replacing an executor then need not erase the durable next step.

Stages may cover topic selection, definition, implementation, execution and adjudication, each with output and review. Cap rework: this case uses three rounds to prevent indefinite looping, not because three rounds universally establish correctness.

2. No improvement differs from an invalid experiment

Result	Meaning	Next action
WIN	Improvement observed	Retain result and comparison evidence
NULL	No improvement observed	Record a tested unhelpful direction to avoid repetition
FAIL	Performance worsened	Retain failure evidence and revert to a usable result
DRIFT	Conditions invalidated comparability	Repair conditions, then decide whether to rerun

Calling both NULL and DRIFT failure loses the distinction between an ineffective direction and one not yet validly tested. Likewise, preserve a previous successful capture when a new fetch fails; do not overwrite it with an empty file.

3. Four constraints for long-running processes

Repeated triggers: heartbeats and schedules should be idempotent so another trigger does not start the same task again.
Concurrency: cap in-flight work and candidates. The case's concurrency of two and replenishment below five in-flight tasks up to a ceiling of twenty are illustrative watermarks.
Claiming and reclamation: record the executor, last activity and timeout. Establish whether an executor is still active before reclaiming slow work.
Artifact replacement: write temporary results, require success and nonempty checks, then replace the official artifact. Append-only status logs support replay and stage timing.

4. Unattended does not mean unlimited authority

Prompts may allow continued execution of already-authorized work and define exit states. The case's “just do it” instruction applies only to bounded background work; it cannot replace authorization for publication, payments or permission changes. Dangerous controls default off; project policy governs reversible low-impact actions.

For rate limiting, make failures observable and preserve progress before a fallback, avoiding repeated external effects. “Kill and restart whenever limited” is not an unconditional rule.

5. Use retrospectives to reduce repetition

Keep completed and rejected lists. Decide whether a candidate warrants effort before expensive execution and review. Attach a plain-language explanation to each artifact so people can judge its purpose quickly.

Monitor a symptom that reveals a blocked stage. VoidSpark focuses on idle GPUs; a content process could watch for new material without valid output. Such a signal helps locate a bottleneck but does not replace cost, safety or quality constraints.

Addition: Test decision components separately and choose thresholds by error cost

This addition is based on the October 16 Builders account of the interview with Diogo Almeida about Jev. It records the guest’s methods and supplier claims without reproducing product benchmarks. The existing discussion of type safety, probability calibration and authorization remains unchanged.

An email assistant can separately assess whether a message concerns sales, recruiting or a request for the recipient’s personal attention, then combine those judgments in code. Separate tests make failures easier to locate and retain as regression examples. More modules alone are not evidence of better design. The combined workflow still needs tests for overlapping conditions, precedence and uncertain cases. These are proposed applications, not a claim that the interview solved email scheduling.

Missing an important message and reviewing an extra ordinary message have different costs. Specify those costs before choosing thresholds, then observe both missed targets and false alarms on actual samples. A threshold expresses a trade-off; reducing alerts does not by itself demonstrate improved accuracy.

Almeida describes RLCD as involving input and output design, training data, balancing and objectives, rather than merely a different reinforcement-learning algorithm. He says the team builds its own data and combines open-weight models instead of pretraining from scratch. Check whether data covers relevant boundary cases, whether objectives reflect costly errors and whether results hold in the actual workflow. The account does not provide a complete reproducible training recipe or independent performance ranking.

Use this approach when one prompt handles many decisions and failures are difficult to isolate. Start with judgments that can be clearly labeled and compare maintenance costs. For low-volume tasks with clear rules, deterministic code may be sufficient.

Added 2026-10-18: Connecting Environment, Feedback, Collaboration, and Responsibility into a Deliverable Agent

Several 42 Zhangjing interviews add a product perspective to the orchestration and verification discussed above: from Dify in 2023 to Pokee and Grasp in 2025, then Huang Dongxu’s practical work, Junior, and Slock in 2026. These are practices from different periods and tasks, not an established, unified technical path. Historical capabilities and prices are not current benchmarks.

Before Deciding the Next Step, Give the Task Observable State

First define the goal, current state, available actions, outcome feedback, and fallback after failure. A missing API, a supplier who only takes phone calls, or an expired login session can prevent an otherwise sound plan from being completed. A model’s ability to express a plan does not mean the environment can execute it, much less that the result has been delivered.

Pokee separates the decision core built on reinforcement learning (RL, learning policies from action feedback) from the large language model (LLM, understanding language and generating expression). Its criteria for applicability are more useful than the claim that “agents must use RL”: does the task require exploring several unknown actions, do earlier steps affect later resources, and can long-term outcomes be observed and improved? Wu Yi’s interview on reinforcement learning emphasizes the complementary roles of understanding the environment and making sequential decisions, including the example of first asking for the environment version when information is incomplete. Application teams without reliable environments, acceptance criteria, and samples should fill those gaps first; they need not begin by training a model.

Different approaches retain different boundaries. Rules suit stable processes, model planning suits steps requiring judgment, and training suits repeated tasks with sufficient feedback. A single task can combine them. When an eleven-step path fails, the system needs to know which step it can return to and which external states have already changed. Generating the plan again does not erase side effects.

An Environment Includes Recovery and a Controllable Scope of Impact, Not Just a Tool List

Grasp distinguishes the runtime environment, decision loop, domain knowledge, and acceptance: a page loading and a task being completed are different things. Its state-preservation example focuses on resuming after an hour-long interruption and consuming fewer resources while idle. Huang Dongxu packages skills, environment state, and context together. Combining these ideas suggests that a reusable unit should include at least its inputs, tools and versions, login and permissions, data locations, progress cursor, evidence of completion, and failure recovery.

The interviews disagree about cloud and local deployment. Lei Lei places greater weight on cloud elasticity and continuous operation; Huang considers both local and cloud execution viable, with control and maintenance as considerations. Neither prediction should become the only deployment answer. The practical choice depends on data requirements, observability, cost, task duration, recovery after downtime, and who can stop execution.

A sandbox—an environment limiting which files, networks, and systems code can affect—limits the worst consequences; it does not make every action inherently safe. A login module being able to use credentials is also different from putting a password into model context. Restoring a session requires confirming that identity and permissions remain valid, not merely that the process is still alive.

Organizational and Personal Memory Serve Different Audiences and Have Different Sharing Boundaries

Junior emphasizes that an enterprise employee is not the owner’s alter ego. It needs to understand the company, projects, roles, and responsibilities, but an agent knowing everything about a small team should not be directly extended to an organization of ten thousand people. Remembering the boss’s private conversation does not authorize repeating it in a group, and knowing customer information does not mean every salesperson may access it.

Memory usability therefore involves at least two questions: does this information help the current task, and is the current recipient entitled to receive it? Identity, project, time, information source, and permitted sharing scope should be stored alongside the content. Role labels help guide behavior; they are not enforceable permissions. Access to tools and data requires system state and auditable constraints.

Gaotu’s case offers another use of memory: a lead’s history, intent, trial lessons, and questions form a report that is handed to a human to close the sale. It shows that complete context first serves handoff quality, without requiring the agent to know every detail of the customer’s life.

Reduce Duplicate Claims and Lost Context Before Pursuing More Multi-Agent Parallelism

Slock notes that people can see where an old thread belongs, while a model receives a mixed stream of events. A message ID or an @mention alone may not tell it which matter is under discussion or which identity it is acting under. When entering a work thread, supplying its topic, a summary of relevant earlier messages, the current owner, and the next action is more targeted than putting the entire group conversation into one context.

Task claims need an exclusive state: who accepted the task, when work began, where the artifact is, and who will accept it. Merely saying “do not duplicate work,” without shared task state, can still leave several agents editing the same file. Huang describes rapid changes to the same module overwriting one another. Parallelism can begin with work whose module boundaries are clear and whose results can be independently verified. The lead agent also needs to reconcile actual artifacts rather than simply adding together every sub-agent’s self-reported completion.

Reminders among agents may improve correction, but can also spread the same mistake. Slock’s observation of one corrected agent reminding another is an observation about collaboration, not proof that multi-agent consensus is necessarily reliable. Architecture comparisons should include context duplication, waiting, conflicts, review, and token costs, judging the benefit by total time and cost per acceptable delivery.

Evaluate Completion, Regression, Clarification, and Refusal to Exceed Boundaries

Ding Ding’s discussion of evaluation takes real input, complete output, and business criteria as the minimum test unit. Public leaderboards assess general capabilities; an internal task set assesses the business. Neither should replace the other. End-to-end cases can use the following dimensions:

Dimension	What to observe	Example failure
Understanding and clarification	Identify missing information and ask critical questions promptly	Repeatedly making incorrect changes without asking for the environment version
Actions and state	Correct tools, parameters, and verification by reading back state	The page reports success, but no order or file was created
Results and handoff	The artifact meets requirements and can be checked and continued by the next person	A polished summary without supporting evidence or output
Regression and coverage	Samples cover old and new tasks, different user groups, and the long tail	A new capability scores higher while an existing core process deteriorates
Permissions and confidentiality	Do not share unauthorized information or take unauthorized actions	Citing the boss’s private conversation while answering a group question
Recovery and correction	Stop after an error, explain its impact, and retain a correction record	Retrying causes a duplicate send, or missing data is silently replaced with zero

Junior retains both a fixed regression set and a path for frontline staff to explore new models. Preventing degradation on old questions and discovering new uses are different activities. Held-out, unseen samples help reduce teaching to the test, while version comparability must also be maintained. Automated judges should be cross-calibrated with humans and online behavior; a model’s “pass” is not an objective fact.

Character.AI’s post-training discussion provides a further counterexample in product metrics. Average conversation duration may increase because the model keeps asking questions to avoid sensitive topics, while heavy users’ experience declines. Edits, deletions, and regenerations can help construct preference samples, but their context and reasons must be retained. Not every edit means a preference for a particular writing style. Training use must also remain within the data’s original permitted scope. The quality of a long-term relationship, pleasing someone in a single turn, and short-term engagement should be tested separately.

Keep Responsibilities Explicit Between Judgment, Verification, and Action

The Vibe-Trading interview separates the model producing hypotheses, the harness—a system constraining execution and verification—providing checks, and the human owning scope and exceptions. Its most transferable elements are checks on data definitions, public availability dates, versions, and missing data. A thorough explanation does not guarantee trading profit, and “making money” is not the sole criterion of code correctness.

Actions that change business data can follow proposal → preview → execute → readback, with each action tied to its evidence, scope, and acceptable loss. Insufficient information should explicitly stop the process at “cannot determine.” A failed data source should be marked as missing, not silently filled with zero. A self-correction report shows that one error was found; it does not exempt future work from verification. Multiple judgments sharing the same incorrect source can fail together, so their individual confidence levels cannot simply be multiplied.

Cost and Operations Are Part of Architectural Acceptance

Junior encountered tasks with the same name taking ten to thirty rounds under fixed per-task pricing, and later changed its billing and user entry points. Low-priced student materials generated high exposure without covering service costs. The lesson is to count the complete multi-round cost, not just a single API call. Dify emphasizes that operators must keep materials, tools, and prompts updated, and rerun business tests after replacing a model.

The application sequence proposed here is: first use one real task to validate information → action → readback and define permissions, recovery, and cost; then expand the task set, parallelism, and autonomy. This is an editorial synthesis of mechanisms described in the interviews. It does not mean these products have been independently validated, nor require every task to use the same model, RL, or multi-agent architecture.

Tool Reuse and Process Visibility Still Need Data Completeness

Sheet0’s Wang Wenfeng emphasizes that context—the information available to the current task—includes code structures, web pages, tables, and tool execution results, with each round of feedback changing the next step. Tested query and execution components should be reused, leaving open-ended judgment where it is needed. But “the script ran,” “the table has no empty cells,” and “we retained code that succeeded once” do not establish 100% correctness across all inputs. Collection tasks must also check start and end boundaries, page counts, unique IDs, totals, reasons for missing data, and field sources. Publication tasks must verify actual storage and the public result. Seeing the browser operate can help users retain control, but cannot replace this evidence of completion.

Gru’s Zhang Hailong discusses a narrow entry point for a unit-testing agent and a multi-repository task set, showing how repetitive labor can have well-defined inputs and outputs. Generated tests, however, may enshrine existing bugs as expected behavior. Intended results therefore need independent definitions. Coverage and pass rates are meaningful only if the tests themselves are correct.

Revisions Across Interviews with the Same Product: Accuracy Claims Are Not Acceptance Criteria

Sheet0’s 2025 interview emphasized stable, complete collection and processing of structured data. When Wang Wenfeng returned in March 2026, he explicitly acknowledged at 51:51 that the earlier pursuit of 100% accuracy was beyond what the model could actually achieve, and that it was not users’ main bottleneck at that time. This change should be read alongside the earlier claim. Result accuracy still needs acceptance testing, but product priorities must follow the obstacles in real tasks rather than turn an earlier marketing statement into a technical guarantee. An actionable approach is to record missing rows, incorrect columns, missing fields, unavailable data, and items requiring human confirmation separately, state a verifiable scope of completion, and prioritize fixes by user loss. The 2026 return interview

Stable and Replaceable Layers of Engineering Assets

Dify’s return interview in December 2025 traced the evolution of its 2023 prompt/RAG tools into enterprise workflows, plugins, and coordination across multiple models. Model upgrades should not force users to rebuild every process. Plugins, retrieval components, and model interfaces can be replaceable, while sensitive business fields, approved steps, and existing assets need compatibility commitments. The guest’s description of being “production-ready” does not refer to one accuracy figure, nor mean zero failures. Concrete acceptance checks should cover regression of existing workflows, field structures, stopping at exceptions, human–AI review, upgrades, and rollbacks. Four or five hundred nodes, or thousands of workflows, are self-reported customer examples; scale does not establish that fully automated management has been achieved. For nonstandard collaboration between organizations, a workflow’s role is to express clearly who executes, who provides feedback, and where a human must be involved. When a person repeatedly generates and selects a satisfactory result, delivery responsibility still rests with that person. This and the early Dify notes from 2023 describe an evolution across stages; later capabilities must not be retroactively attributed to the initial version. Dify’s 2025 return interview

Addition: For natural-language-to-SQL, validate business meaning before production use

This section draws on the Builders account of Andy Pavlo’s MAD interview. It distinguishes the guest’s experience from the applications proposed here. It adds concrete checks for database tasks. One bank’s investment or accuracy is not a universal standard, and the interview’s forecasts of future traffic are not treated as established facts.

Map business terms to query constraints that people can review

An executable SQL statement only shows that the database accepted it. Whether its result answers the business question also depends on definitions and data scope. Before generation, establish the business terms, corresponding tables and columns, relationships, time conventions, access permissions and what counts as the correct result. A model or MCP interface having database access does not mean it knows the company’s business semantics or may read every column.

The following are proposed clarification and acceptance questions, rather than implementations supplied by the interview:

Check	Clarify before generation	Retain for acceptance
Business definition	Does “active customer” mean a login, a transaction or a contract?	An agreed definition and counterexamples
Tables and columns	Which table and primary key apply? How should records be joined and deduplicated?	Column mappings and known correct results
Data scope	What are the time boundaries, time zone and permitted data scope?	The data version, scope and authorization record
Result meaning	Should the query return individual records, a count of people or a monetary amount? How are missing values interpreted?	Human-reviewed examples and expected results

Record execution, result correctness and production validation separately

First validate meaning against internal business tasks with explicit expected results, then check whether it holds with real data and permissions. A run without errors, a matching result on one example and sustained usability in production represent different levels of acceptance. Keep failed questions, incorrect SQL, expected results and the regression examples used after a fix together. These records help distinguish failures in business definitions, column mappings, data scope and generation.

Pavlo warns that training data may cover public benchmarks, so benchmark scores cannot directly stand in for a particular company’s actual accuracy. This is a risk raised in the interview, not independent proof that a particular model’s evaluation was contaminated. Record test and production workloads separately as well. Passing in a test environment still requires checking production data scale, query behavior and actual business results; test success alone does not establish production acceptance.

Use this approach when business users ask questions in natural language, agents generate queries or analysis spans multiple tables. If business definitions are unsettled, first supply those definitions and examples of correct results. A model with a higher leaderboard score cannot resolve an unspecified business definition.

October 21, 2026 addition: persistent work does not require an always-on execution computer

This addition is based on Shensiquan’s Chinese account of Lee Robinson’s Stanford presentation. The original course, Grok Bot code and deployment have not been independently checked. Claims that hallucinations are essentially solved, automated review is usually safer or agents will soon work continuously for months are not adopted as verified conclusions.

Recover task state separately from the execution computer. The account describes keeping conversations, memory and routines on the server while starting the execution computer on demand and restoring or rebuilding it from snapshots. Patching, replacing or suspending that computer need not discard the task’s goal and progress. Firecracker and Temporal are named in the source. The transferable mechanism is a replaceable execution environment with reliable records of progress and results, rather than a requirement to use those particular technologies.

Recovery needs separate checks for where to resume and whether the previous external action already took effect. Durable workflows can help preserve progress, but a framework name alone does not guarantee that an email, payment or external write happens exactly once. When a result is uncertain, inspect external state and receipts before retrying. Isolation, the task loop and identity permissions also require separate verification.

Prioritize user interruptions and combine background events. The account gives direct changes to a user’s goal priority, while group messages and background work queue up and may be batched when they arrive together. Preserve each event’s source and handling result so batching does not lose a cancellation or changed instruction. Remaining quiet when there is no material progress differs from stopping execution. Notifications, requests for approval and messages already sent should have distinct records.

Keep cache prefixes stable while recognizing compression loss. Stable ordering of tool definitions and system prompts can reduce unnecessary prefix changes. Preparing a compressed summary while a long conversation’s cache is warm is also a cost-management idea. Cache hits still depend on the service, retention period and request content; not every miss has the same cause. Compression loses information, so retain original logs, essential constraints and retrievable files rather than treating a summary as a complete original record.

October 23, 2026 Addition: Estimate Agent Work From Its Execution Model

Estimate delegated work from how the agent actually executes it, rather than copying a human working-day estimate. Swyx’s personal assessment of Opus 5.5 stresses that sustained execution and parallelism can change elapsed time. His example of two or three days versus two or three hours is not a universal productivity guarantee and comes without an independent benchmark.

Application in this knowledge base: ask the agent to identify subtasks, dependencies, parallel work and human handoffs before estimating completion. Update remaining time and its reasons when blockers or repairs appear. After completion, compare the forecast with elapsed time to learn which tasks this model and tool combination estimates well. A record of “initial estimate—latest estimate—actual time—reason for error” is an editorial suggestion, rather than a format specified by the source.

A short estimate does not replace authorization or remove time spent waiting for a person, service queue or unavailable tool. External commitments still depend on actual dependencies and resources. Link this to planning, isolated execution and independent review in the coding-agent article; estimation quality does not prove output correctness.

Source: https://x.com/swyx/status/2113283272130412684

Addition: Check logical isolation and shared-hardware risks separately

Separating agents’ accounts, files, processes and execution sandboxes does not establish that they have no shared underlying hardware. Guillermo Rauch’s account of a shared-host timing issue adds a different layer to isolation checks: when two actors share a machine, CPU-cache interference may be observable through their own execution timing. Such a side channel does not require directly handing one task’s files to another. A check that directories cannot access each other therefore cannot establish complete isolation.

This retains Rauch’s public account; the attack has not been independently reproduced here, and Vercel Sandbox has not been audited. He presents dedicated hardware per agent as addressing this class of shared risk, while acknowledging its economic cost. That must not be expanded into a claim that every attack is solved. Ask separately how logical access is isolated, whether machines and caches are shared, what mitigations and monitoring are used, and what residual risks remain. A vendor’s report that mitigations are deployed is different evidence from proof that a risk is eliminated.

Editorial application: select an execution arrangement according to task sensitivity and cost, and record the shared layers, known limits and incident owner during acceptance. Dedicated hardware still does not replace account permissions, code review or readback of external actions. Low-risk demonstrations need not copy the most expensive configuration. For related operational responsibilities, see Enterprise AI.

Source: Guillermo Rauch’s post. The acceptance checks above are editorial applications.