--- title: "Why multi-agent LLM systems fail: 14 MAST failure modes | Len P. van der Hof" description: "The MAST study labelled 1,642 traces and found 14 failure modes. Separate the agent's mistake from the workflow that let it pass, then test one failed run." image: "https://lenvanderhof.com/media/generated/blog-hero-multi-agent-failure-modes-explained-v1.cc2d29fe31e5.wide.webp" --- [AI Systems](https://lenvanderhof.com/en/blog/category/ai-systems/) Research guide # Why multi-agent LLM systems fail: the 14 MAST failure modes An agent's mistake and the workflow that let it through are two different problems. MAST gives you a vocabulary for both. Len P. van der HofPublished 15 October 202610 min read Both runners were fast. The baton is still on the track. Direct answer Multi-agent LLM systems fail when an agent makes a mistake and the workflow around it lets the mistake pass: roles are unclear, context is lost or withheld between agents, or checks accept unfinished work. The MAST study (Cemri and colleagues, NeurIPS 2025) labelled 1,642 execution traces from seven open-source frameworks and found 14 failure modes in three groups: system design issues, inter-agent misalignment and task verification. Clearer role prompts and a review loop helped in the authors' tests but did not establish reliability. The figures describe research frameworks on benchmark tasks, not your company's failure rate. ## Key takeaways - Diagnose a failed run on two levels: what the agent got wrong, and what the workflow let through. Each level calls for a different fix. - MAST names 14 failure modes in three groups: system design issues (44.2% of labelled failures), inter-agent misalignment (32.3%) and task verification (23.5%). - The largest single modes were step repetition (15.7%), reasoning-action mismatch (13.2%) and being unaware of termination conditions (12.4%). - Failure rates of 41% to 86.7% were measured on research frameworks and benchmark tasks, not on company projects. - Clearer role prompts lifted ChatDev from 25.0% to 34.4% success on 32 tasks; a separate review-loop intervention reached 40.6%. The authors did not call either a substantial improvement. - The Multi-Agent Organization maps the three groups onto CHORUS: Charter for design, Handoff and Orchestrate for misalignment, Review for verification. MAST did not test CHORUS. **A multi-agent failure has two parts: the mistake an agent made, and the workflow that let it pass.** Diagnose them separately. Look at what the agent did, what information reached it, and what the next handoff or review should have caught. That split is a useful lens for “Why Do Multi-Agent LLM Systems Fail?” by Mert Cemri, Melissa Z. Pan, Shuyi Yang and colleagues at UC Berkeley and Intesa Sanpaolo, published at NeurIPS 2025 ([arXiv 2503.13657](https://arxiv.org/abs/2503.13657)). The authors sorted the failures they observed into 14 failure modes in three groups and named the taxonomy MAST, the Multi-Agent System Failure Taxonomy. A multi-agent LLM system uses several [AI agents](https://lenvanderhof.com/glossary/ai-agent/), each powered by a large language model and given its own role, to complete one task together. ChatDev, one of the systems studied, simulates a software company with a CEO, a CTO, a programmer, a reviewer and a tester, all played by AI. ## What did the study actually do? The team collected runs from seven open-source frameworks: ChatDev, MetaGPT, HyperAgent, AppWorld, AG2 (formerly AutoGen), Magentic-One and OpenManus. The tasks were coding, maths and general assistant jobs. Each run produced a *trace*: the full log of what every agent said and did. In the first batch, a trace averaged more than 15,000 lines of text. Six experts read over 150 traces and developed the failure categories from what they observed. Three annotators then labelled traces independently, over repeated rounds of refinement, and reached a Cohen’s kappa of 0.88 (a score where 1 means perfect agreement and 0 means chance). An AI judge built on OpenAI’s o1 model labelled the full set of 1,642 traces; its agreement with the human experts was 0.77. These scores measure agreement on labels, not certainty that every diagnosis is correct. On their benchmark tasks, the systems failed between 41% and 86.7% of the time. Those rates describe research frameworks running test tasks with models such as GPT-4o and Claude 3.7 Sonnet. They are not a failure rate for your company. ## The 14 failure modes, explained through a kitchen Picture a restaurant kitchen where every worker is an AI agent. The failure-mode names come from the paper; the kitchen scenes are this article’s illustrations. The last column shows each mode’s share of all failures labelled across the 1,642 traces, not the share of runs that failed. CodeFailure modeWhat it meansIn the kitchenShareSystem design issues44.2%1.1Disobey task specificationIgnores the task’s constraints or requirementsThe ticket says no nuts. The dish has nuts.11.8%1.2Disobey role specificationActs outside its role, or like another agentThe dishwasher starts plating desserts1.5%1.3Step repetitionRedoes steps that were already finishedThe same sauce gets made twice because nobody ticked it off15.7%1.4Loss of conversation historyLoses recent context and falls back to an earlier stateA cook forgets that table six changed its order and cooks the original2.8%1.5Unaware of termination conditionsDoes not recognise when the job is doneThe dish is finished but keeps being “improved” until it is cold12.4%Inter-agent misalignment32.3%2.1Conversation resetThe dialogue restarts and progress is lostMid-service, the ticket rail is wiped and everyone starts over2.2%2.2Fail to ask for clarificationCarries on with unclear information instead of askingNobody asks “medium or well done?” and the steak goes out wrong6.8%2.3Task derailmentDrifts away from the actual goalA cook starts designing a new menu during the lunch rush7.4%2.4Information withholdingHas information another agent needs but does not pass it onThe sous-chef knows the fish ran out and says nothing0.85%2.5Ignored other agent’s inputDisregards what another agent saidThe waiter reports an allergy and the grill ignores it1.9%2.6Reasoning-action mismatchTakes an action inconsistent with its stated reasoning”No salt for table four,” says the cook, and salts it13.2%Task verification23.5%3.1Premature terminationStops before the goal is met or the needed information is exchangedThe plate leaves the pass before the sauce is on6.2%3.2No or incomplete verificationThe check is missing or partialNobody tastes it before it goes out8.2%3.3Incorrect verificationThe check happens, but checks the wrong thing or misses the errorThe head chef confirms the plate looks nice, not that the chicken is cooked9.1% Read the table with the opening split in mind. Some modes describe an agent’s own mistake: in 2.6, the action contradicts the reasoning the agent just gave. Others describe what the workflow failed to carry or catch: in 2.4, information never reaches the agent that needs it; in 3.2, the check is missing or partial. One failed run can contain both: an agent ignores a constraint (1.1), and a check that looks at the wrong thing lets it through (3.3). ## Three failures from the paper itself **The Wordle game (1.1).** Asked for a Wordle game with a new five-letter word every day, ChatDev produced a program with a fixed word list. Even after explicit instructions against a fixed word bank, it still used one and added new errors. The authors treat this as a problem in how the system handles instructions, not only a limit of the model. **The phone login (2.4).** In one trace, a phone agent knew that the username had to be a phone number but did not pass that on to the supervisor agent. The supervisor did not ask. Login attempt after login attempt failed, and so did the task. **The chess program (3.2).** ChatDev built a chess game that passed its checks because the code compiled, yet runtime bugs made it unusable. The checks did not test whether it followed the rules of chess. The paper describes verifiers that checked whether code compiled or whether TODO comments were left, without adequately checking whether the program met the task. In all three, an agent got something wrong, and nothing later in the workflow caught it. ## What the authors tested, and what they proposed The authors distinguish *tactical* interventions, such as clearer prompts and roles, better conversation patterns and self-checks, from *structural* changes: stronger verification, standard ways for agents to communicate, confidence thresholds (an agent acts only when it is confident enough), and better memory and state management. These are directions for improvement, not a proven recipe for reliability. On 32 programming tasks, ChatDev’s baseline succeeded 25.0% of the time. Clearer role prompts, including a rule that only a senior agent may end a conversation, raised success to 34.4%. A separate intervention rebuilt the workflow around a review loop that ends only when the CTO agent confirms the reviews are satisfied; it reached 40.6%. Each figure is a comparison with the baseline, not a gain to add to the other. The authors’ own verdict: the interventions “do not constitute substantial improvements.” One more distinction matters if you are buying tools: message format versus message content. The paper discusses standards such as MCP and Agent-to-Agent (A2A), yet the authors observed misalignment between agents in the same framework talking in plain language. They trace this to agents failing to model what another agent needs to know. A standard format can still carry an incomplete message. ## What the numbers cannot tell you The table cannot tell you how often your system will fail, or which failure will cost you most. It shows shares of labelled failures in a research corpus, and an AI judge labelled most of those traces. The authors say the taxonomy is not exhaustive. Failure profiles also differed by system: AppWorld often stopped too early; OpenManus often repeated steps. What the labels do give you is a diagnosis specific enough to act on. “The agents were bad” is not a diagnosis. “We have a 2.2 and a 3.3” is a start, provided you can point to the log lines behind each code. Those lines tell you where to intervene. ## CHORUS: one way to organise the fixes **CHORUS** is the coordination protocol from *[The Multi-Agent Organization](https://lenvanderhof.com/books/multi-agent-organization/)*, written for small teams of roughly 2 to 15 people working with AI agents. It is an operating design, not a research finding: MAST did not test it, and the book says MAST does not validate it. Its six skills give you places to put the changes you will test: 1. **Charter.** Give every worker, human or agent, a one-page contract with seven fields: purpose, inputs, outputs, authority ceiling, success criteria, review requirement and escalation. 2. **Handoff.** Send a context packet with six fields across every task boundary (goal, constraints, inputs, provenance, acceptance criteria and deadline), so the receiver does not have to guess. 3. **Orchestrate.** Decide which steps run in sequence, where work fans out to parallel agents and merges back, and which named person takes escalations. 4. **Review.** Let output through a gate only when it meets written acceptance criteria, not because it sounded right. 5. **Update.** Feed review findings back into prompts, tools, SOPs and charters, then check whether the change helped. 6. **Sync.** Keep one shared view of who owns what and what is blocked, read in a short weekly check-in. The book maps system design issues onto Charter, misalignment onto Handoff and Orchestrate, and verification onto Review. Treat that as a working map, not as evidence that those practices resolve every failure in a group. Take a hypothetical three-agent support pipeline: a triage agent sorts the ticket, a drafting agent writes the reply, a checking agent reviews it. The *charter* defines done as one reply, under 150 words, linking the right help article; that control targets 1.5. The *handoff* must carry the customer’s plan and previous tickets, and the drafter is told to stop when either is missing; that is aimed at 2.2 and 2.4. The *review* checks the reply against the refund policy, not just its tone; that is aimed at 3.3. None of these controls is proven until you test it. For the wider argument about why teams get slower when they add agents, read [the coordination gap](https://lenvanderhof.com/en/blog/why-multi-agent-systems-fail/) and [what agent orchestration means](https://lenvanderhof.com/en/blog/what-is-agent-orchestration/). The [CHORUS framework page](https://lenvanderhof.com/frameworks/chorus/) has the full step list. ## Try this today: inspect one failed run in 20 minutes 1. **Pick the failure.** Save the input and the trace of one recent failed run. Write down what an acceptable result would have been. 2. **Find the agent’s mistake.** Locate the first point where the work visibly departs from that requirement. Attach the MAST code and the log lines that support it. Mark an uncertain label as uncertain. 3. **Find the workflow’s gap.** Look at the next boundary. What could the receiving agent or the reviewer see? Was context missing, a constraint dropped, or a check absent? 4. **Change one control.** Clarify a stopping rule, require the missing handoff field, or add a check for this specific error. Write down what the revised workflow must do before you test it. 5. **Replay the saved input.** Check whether the control did its job and whether the final result met the original requirement. Record both: catching an error and completing the task are different outcomes. For the support example, replay a ticket with the customer’s plan missing. The drafter should stop and ask for it before writing. If it guesses the plan and carries on, the handoff control has failed its test. One replay cannot establish reliability. It shows whether one control worked on one run. Keep the trace and the acceptance check so you can run the test again after the next change. Cite this:Why multi-agent LLM systems fail: the 14 MAST failure modes.Len P. van der Hof. [https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/](https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/) · Published 15 October 2026. ## Terminology - [TRACE](https://lenvanderhof.com/glossary/trace/) - [CHORUS](https://lenvanderhof.com/glossary/chorus/) ## Sources 1. [Why Do Multi-Agent LLM Systems Fail? (arXiv 2503.13657, v3)](https://arxiv.org/abs/2503.13657) · Cemri, Pan, Yang, Agrawal, Chopra, Tiwari, Keutzer, Parameswaran, Klein, Ramchandran, Zaharia, Gonzalez, Stoica (UC Berkeley, Intesa Sanpaolo) 2. [Why Do Multi-Agent LLM Systems Fail? (NeurIPS 2025, Datasets and Benchmarks Track)](https://proceedings.neurips.cc/paper_files/paper/2025/file/b1041e52d3be19f0a9bc491657488e4a-Paper-Datasets_and_Benchmarks_Track.pdf) · NeurIPS Proceedings 3. [NeurIPS 2025 poster: Why Do Multi-Agent LLM Systems Fail?](https://neurips.cc/virtual/2025/poster/121528) · NeurIPS 4. [CHORUS (framework)](https://lenvanderhof.com/frameworks/chorus/) 5. [The Multi-Agent Organization](https://lenvanderhof.com/books/multi-agent-organization/) 6. [Why multi-agent systems fail: the coordination gap](https://lenvanderhof.com/en/blog/why-multi-agent-systems-fail/) ## Further reading - [Why multi-agent systems fail: the coordination gap](https://lenvanderhof.com/en/blog/why-multi-agent-systems-fail/) - [CHORUS](https://lenvanderhof.com/frameworks/chorus/) - [The Multi-Agent Organization](https://lenvanderhof.com/books/multi-agent-organization/) - [What is agent orchestration?](https://lenvanderhof.com/en/blog/what-is-agent-orchestration/) About the author ## [Len P. van der Hof](https://lenvanderhof.com/en/authors/len-p-van-der-hof/) Entrepreneur, AI Innovator and Venture Builder Len P. van der Hof builds practical AI systems, digital ventures and evidence-informed tools for founders. ```json { "@context": "https://schema.org", "@graph": [ { "@type": "Person", "@id": "https://lenvanderhof.com/#person", "name": "Len P. van der Hof", "alternateName": [ "Len van der Hof", "L.P. van der Hof", "Leendert Pieter van der Hof" ], "honorificSuffix": "MSc", "url": "https://lenvanderhof.com/", "image": [ "https://lenvanderhof.com/photos/len-portrait-1.jpg", "https://lenvanderhof.com/photos/len-portrait-2.jpg", "https://lenvanderhof.com/photos/len-portrait-3.jpg", "https://lenvanderhof.com/photos/len-portrait-4.jpg", "https://lenvanderhof.com/photos/len-speaking.jpg", "https://lenvanderhof.com/photos/len-hero.jpg" ], "jobTitle": "Entrepreneur, AI Innovator and Venture Builder", "description": "Len P. van der Hof, MSc, is a Dutch entrepreneur and AI innovator in Zwijndrecht. He builds ReasonKit, MindSesh, Undominated.ai, books under his name, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures. Eleven titles in Systems for the Strategic Self are available now, in English and Dutch.", "address": { "@type": "PostalAddress", "addressLocality": "Zwijndrecht", "addressCountry": "NL" }, "alumniOf": { "@type": "CollegeOrUniversity", "name": "Rotterdam School of Management, Erasmus University" }, "knowsAbout": [ "Artificial intelligence", "AI agents", "Agentic AI systems", "LLM routing", "SEO", "Generative engine optimization", "Answer engine optimization", "Venture building", "Founder performance", "Founder psychology", "Evidence-based decision-making" ], "sameAs": [ "https://www.linkedin.com/in/lenvanderhof/", "https://x.com/LenvanderHof", "https://www.youtube.com/channel/UCTG20buKqYYbitqqf7l3zJA", "https://www.instagram.com/Lenvanderhof/", "https://www.threads.com/@lenvanderhof", "https://github.com/Lenvanderhof", "https://huggingface.co/LPH98", "https://www.npmjs.com/~lenvanderhof", "https://www.goodreads.com/author/show/70983905.Len_P_van_der_Hof", "https://www.amazon.com/author/lenvanderhof", "https://www.bol.com/nl/nl/b/len-p-van-der-hof-msc/609879394/", "https://bsky.app/profile/lenvanderhof.com", "https://mastodon.social/@Lenvanderhof", "https://crates.io/users/Lenvanderhof", "https://cursor.com/@Lenvanderhof", "https://medium.com/@Lenvanderhof", "https://gitlab.com/Lenvanderhof", "https://hub.docker.com/u/lenvanderhof/", "https://dev.to/lenvanderhof", "https://www.facebook.com/Lenvanderhof", "https://soundcloud.com/Lenvanderhof" ], "affiliation": [ { "@id": "https://lenvanderhof.com/#publisher" }, { "@id": "https://lenvanderhof.com/#mindsesh" }, { "@id": "https://lenvanderhof.com/#lifestyle" } ] }, { "@type": "WebSite", "@id": "https://lenvanderhof.com/#website", "url": "https://lenvanderhof.com/", "name": "Len P. van der Hof", "description": "Len P. van der Hof, MSc, is a Dutch entrepreneur and AI innovator in Zwijndrecht. He builds ReasonKit, MindSesh, Undominated.ai, books under his name, the fiction imprint LPH98.lifestyle, and technology ventures through LPH98.ventures. Eleven titles in Systems for the Strategic Self are available now, in English and Dutch.", "inLanguage": [ "en", "nl" ], "publisher": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#publisher", "name": "LPH98.ventures", "url": "https://lph98.ventures", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#mindsesh", "name": "MindSesh", "url": "https://mindsesh.net", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#reasonkit", "name": "ReasonKit", "url": "https://reasonkit.sh", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "SoftwareApplication", "@id": "https://lenvanderhof.com/#undominated", "name": "Undominated.ai", "url": "https://undominated.ai", "creator": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "Organization", "@id": "https://lenvanderhof.com/#lifestyle", "name": "LPH98.lifestyle", "url": "https://lph98.lifestyle", "founder": { "@id": "https://lenvanderhof.com/#person" } }, { "@type": "ImageObject", "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#primaryimage", "url": "https://lenvanderhof.com/media/generated/blog-hero-multi-agent-failure-modes-explained-v1.cc2d29fe31e5.wide.webp", "contentUrl": "https://lenvanderhof.com/media/generated/blog-hero-multi-agent-failure-modes-explained-v1.cc2d29fe31e5.wide.webp", "representativeOfPage": true }, { "@type": "BreadcrumbList", "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#breadcrumb", "itemListElement": [ { "@type": "ListItem", "position": 1, "name": "Home", "item": "https://lenvanderhof.com/" }, { "@type": "ListItem", "position": 2, "name": "Blog", "item": "https://lenvanderhof.com/en/blog/" }, { "@type": "ListItem", "position": 3, "name": "AI Systems", "item": "https://lenvanderhof.com/en/blog/category/ai-systems/" }, { "@type": "ListItem", "position": 4, "name": "Why multi-agent LLM systems fail: the 14 MAST failure modes", "item": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/" } ] }, { "@type": "WebPage", "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#webpage", "url": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/", "name": "Why multi-agent LLM systems fail: the 14 MAST failure modes", "description": "The MAST study labelled 1,642 traces and found 14 failure modes. Separate the agent's mistake from the workflow that let it pass, then test one failed run.", "isPartOf": { "@id": "https://lenvanderhof.com/#website" }, "primaryImageOfPage": { "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#primaryimage" }, "breadcrumb": { "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#breadcrumb" }, "inLanguage": "en-GB" }, { "@type": "BlogPosting", "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#article", "mainEntityOfPage": { "@id": "https://lenvanderhof.com/en/blog/multi-agent-failure-modes-explained/#webpage" }, "headline": "Why multi-agent LLM systems fail: the 14 MAST failure modes", "description": "The MAST study labelled 1,642 traces and found 14 failure modes. Separate the agent's mistake from the workflow that let it pass, then test one failed run.", "datePublished": "2026-10-15T14:00:00.000Z", "author": { "@id": "https://lenvanderhof.com/#person" }, "publisher": { "@id": "https://lenvanderhof.com/#person" }, "image": [ "https://lenvanderhof.com/media/generated/blog-hero-multi-agent-failure-modes-explained-v1.cc2d29fe31e5.square.webp", "https://lenvanderhof.com/media/generated/blog-hero-multi-agent-failure-modes-explained-v1.cc2d29fe31e5.landscape.webp", "https://lenvanderhof.com/media/generated/blog-hero-multi-agent-failure-modes-explained-v1.cc2d29fe31e5.wide.webp" ], "articleSection": "AI Systems", "inLanguage": "en-GB" } ] } ```