Why multi-agent LLM systems fail: the 14 MAST failure modes

A multi-agent failure has two parts: the mistake an agent made, and the workflow that let it pass. Diagnose them separately. Look at what the agent did, what information reached it, and what the next handoff or review should have caught.

That split is a useful lens for “Why Do Multi-Agent LLM Systems Fail?” by Mert Cemri, Melissa Z. Pan, Shuyi Yang and colleagues at UC Berkeley and Intesa Sanpaolo, published at NeurIPS 2025 (arXiv 2503.13657). The authors sorted the failures they observed into 14 failure modes in three groups and named the taxonomy MAST, the Multi-Agent System Failure Taxonomy.

A multi-agent LLM system uses several AI agents, each powered by a large language model and given its own role, to complete one task together. ChatDev, one of the systems studied, simulates a software company with a CEO, a CTO, a programmer, a reviewer and a tester, all played by AI.

What did the study actually do?

The team collected runs from seven open-source frameworks: ChatDev, MetaGPT, HyperAgent, AppWorld, AG2 (formerly AutoGen), Magentic-One and OpenManus. The tasks were coding, maths and general assistant jobs.

Each run produced a trace: the full log of what every agent said and did. In the first batch, a trace averaged more than 15,000 lines of text. Six experts read over 150 traces and developed the failure categories from what they observed.

Three annotators then labelled traces independently, over repeated rounds of refinement, and reached a Cohen’s kappa of 0.88 (a score where 1 means perfect agreement and 0 means chance). An AI judge built on OpenAI’s o1 model labelled the full set of 1,642 traces; its agreement with the human experts was 0.77. These scores measure agreement on labels, not certainty that every diagnosis is correct.

On their benchmark tasks, the systems failed between 41% and 86.7% of the time. Those rates describe research frameworks running test tasks with models such as GPT-4o and Claude 3.7 Sonnet. They are not a failure rate for your company.

The 14 failure modes, explained through a kitchen

Picture a restaurant kitchen where every worker is an AI agent. The failure-mode names come from the paper; the kitchen scenes are this article’s illustrations. The last column shows each mode’s share of all failures labelled across the 1,642 traces, not the share of runs that failed.

Code	Failure mode	What it means	In the kitchen	Share
System design issues				44.2%
1.1	Disobey task specification	Ignores the task’s constraints or requirements	The ticket says no nuts. The dish has nuts.	11.8%
1.2	Disobey role specification	Acts outside its role, or like another agent	The dishwasher starts plating desserts	1.5%
1.3	Step repetition	Redoes steps that were already finished	The same sauce gets made twice because nobody ticked it off	15.7%
1.4	Loss of conversation history	Loses recent context and falls back to an earlier state	A cook forgets that table six changed its order and cooks the original	2.8%
1.5	Unaware of termination conditions	Does not recognise when the job is done	The dish is finished but keeps being “improved” until it is cold	12.4%
Inter-agent misalignment				32.3%
2.1	Conversation reset	The dialogue restarts and progress is lost	Mid-service, the ticket rail is wiped and everyone starts over	2.2%
2.2	Fail to ask for clarification	Carries on with unclear information instead of asking	Nobody asks “medium or well done?” and the steak goes out wrong	6.8%
2.3	Task derailment	Drifts away from the actual goal	A cook starts designing a new menu during the lunch rush	7.4%
2.4	Information withholding	Has information another agent needs but does not pass it on	The sous-chef knows the fish ran out and says nothing	0.85%
2.5	Ignored other agent’s input	Disregards what another agent said	The waiter reports an allergy and the grill ignores it	1.9%
2.6	Reasoning-action mismatch	Takes an action inconsistent with its stated reasoning	”No salt for table four,” says the cook, and salts it	13.2%
Task verification				23.5%
3.1	Premature termination	Stops before the goal is met or the needed information is exchanged	The plate leaves the pass before the sauce is on	6.2%
3.2	No or incomplete verification	The check is missing or partial	Nobody tastes it before it goes out	8.2%
3.3	Incorrect verification	The check happens, but checks the wrong thing or misses the error	The head chef confirms the plate looks nice, not that the chicken is cooked	9.1%

Read the table with the opening split in mind. Some modes describe an agent’s own mistake: in 2.6, the action contradicts the reasoning the agent just gave. Others describe what the workflow failed to carry or catch: in 2.4, information never reaches the agent that needs it; in 3.2, the check is missing or partial. One failed run can contain both: an agent ignores a constraint (1.1), and a check that looks at the wrong thing lets it through (3.3).

Three failures from the paper itself

The Wordle game (1.1). Asked for a Wordle game with a new five-letter word every day, ChatDev produced a program with a fixed word list. Even after explicit instructions against a fixed word bank, it still used one and added new errors. The authors treat this as a problem in how the system handles instructions, not only a limit of the model.

The phone login (2.4). In one trace, a phone agent knew that the username had to be a phone number but did not pass that on to the supervisor agent. The supervisor did not ask. Login attempt after login attempt failed, and so did the task.

The chess program (3.2). ChatDev built a chess game that passed its checks because the code compiled, yet runtime bugs made it unusable. The checks did not test whether it followed the rules of chess. The paper describes verifiers that checked whether code compiled or whether TODO comments were left, without adequately checking whether the program met the task.

In all three, an agent got something wrong, and nothing later in the workflow caught it.

What the authors tested, and what they proposed

The authors distinguish tactical interventions, such as clearer prompts and roles, better conversation patterns and self-checks, from structural changes: stronger verification, standard ways for agents to communicate, confidence thresholds (an agent acts only when it is confident enough), and better memory and state management. These are directions for improvement, not a proven recipe for reliability.

On 32 programming tasks, ChatDev’s baseline succeeded 25.0% of the time. Clearer role prompts, including a rule that only a senior agent may end a conversation, raised success to 34.4%. A separate intervention rebuilt the workflow around a review loop that ends only when the CTO agent confirms the reviews are satisfied; it reached 40.6%. Each figure is a comparison with the baseline, not a gain to add to the other. The authors’ own verdict: the interventions “do not constitute substantial improvements.”

One more distinction matters if you are buying tools: message format versus message content. The paper discusses standards such as MCP and Agent-to-Agent (A2A), yet the authors observed misalignment between agents in the same framework talking in plain language. They trace this to agents failing to model what another agent needs to know. A standard format can still carry an incomplete message.

What the numbers cannot tell you

The table cannot tell you how often your system will fail, or which failure will cost you most. It shows shares of labelled failures in a research corpus, and an AI judge labelled most of those traces. The authors say the taxonomy is not exhaustive. Failure profiles also differed by system: AppWorld often stopped too early; OpenManus often repeated steps.

What the labels do give you is a diagnosis specific enough to act on. “The agents were bad” is not a diagnosis. “We have a 2.2 and a 3.3” is a start, provided you can point to the log lines behind each code. Those lines tell you where to intervene.

CHORUS: one way to organise the fixes

CHORUS is the coordination protocol from The Multi-Agent Organization, written for small teams of roughly 2 to 15 people working with AI agents. It is an operating design, not a research finding: MAST did not test it, and the book says MAST does not validate it. Its six skills give you places to put the changes you will test:

1. Charter. Give every worker, human or agent, a one-page contract with seven fields: purpose, inputs, outputs, authority ceiling, success criteria, review requirement and escalation.
2. Handoff. Send a context packet with six fields across every task boundary (goal, constraints, inputs, provenance, acceptance criteria and deadline), so the receiver does not have to guess.
3. Orchestrate. Decide which steps run in sequence, where work fans out to parallel agents and merges back, and which named person takes escalations.
4. Review. Let output through a gate only when it meets written acceptance criteria, not because it sounded right.
5. Update. Feed review findings back into prompts, tools, SOPs and charters, then check whether the change helped.
6. Sync. Keep one shared view of who owns what and what is blocked, read in a short weekly check-in.

The book maps system design issues onto Charter, misalignment onto Handoff and Orchestrate, and verification onto Review. Treat that as a working map, not as evidence that those practices resolve every failure in a group.

Take a hypothetical three-agent support pipeline: a triage agent sorts the ticket, a drafting agent writes the reply, a checking agent reviews it. The charter defines done as one reply, under 150 words, linking the right help article; that control targets 1.5. The handoff must carry the customer’s plan and previous tickets, and the drafter is told to stop when either is missing; that is aimed at 2.2 and 2.4. The review checks the reply against the refund policy, not just its tone; that is aimed at 3.3. None of these controls is proven until you test it.

For the wider argument about why teams get slower when they add agents, read the coordination gap and what agent orchestration means. The CHORUS framework page has the full step list.

Try this today: inspect one failed run in 20 minutes

1. Pick the failure. Save the input and the trace of one recent failed run. Write down what an acceptable result would have been.
2. Find the agent’s mistake. Locate the first point where the work visibly departs from that requirement. Attach the MAST code and the log lines that support it. Mark an uncertain label as uncertain.
3. Find the workflow’s gap. Look at the next boundary. What could the receiving agent or the reviewer see? Was context missing, a constraint dropped, or a check absent?
4. Change one control. Clarify a stopping rule, require the missing handoff field, or add a check for this specific error. Write down what the revised workflow must do before you test it.
5. Replay the saved input. Check whether the control did its job and whether the final result met the original requirement. Record both: catching an error and completing the task are different outcomes.

For the support example, replay a ticket with the customer’s plan missing. The drafter should stop and ask for it before writing. If it guesses the plan and carries on, the handoff control has failed its test.

One replay cannot establish reliability. It shows whether one control worked on one run. Keep the trace and the acceptance check so you can run the test again after the next change.

Cite this:Why multi-agent LLM systems fail: the 14 MAST failure modes.Len P. van der Hof. https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/ · Published 15 October 2026.