A Guth Labs publication ↗
Back to News

Anthropic releases benchmark for agents that must share resources

Anthropic announced the release of MAMUBench on November 11, giving developers a way to test what happens when different users’ AI assistants compete for the same budget, calendar or group booking. The accompanying research finds that assigning everyone a separate agent can produce worse collective results than using one coordinator. Anthropic announcement

The announcement is new, but the underlying study is older: its arXiv record dates the first submission to September 30. This is a benchmark-release story, rather than a new frontier-model launch or a claim that the research first appeared this week. Paper record

Researchers examined five models across 77 simulated scenarios. Agents managed shared computing funds, clinic appointments, collective orders or reservations, and a software merge queue. The comparison separated a single coordinator from per-user teams that could communicate and teams that could not. These tasks expose a problem distinct from ordinary delegation: the agents represent different people whose priorities can compete. Research paper

For Claude Opus 5 and Sonnet 5, the coordinator’s average group score exceeded the communicating teams’ score by approximately 10 to 45 percentage points across the four environments. Those scores measure outcomes relative to an optimal strategy, not general model accuracy. In group-planning tests, a coordinator honored a targeted request roughly twice as often. The researchers traced failures partly to agents committing before reading their peers’ messages. Reported findings

The public repository contains 74 scenarios across the API-budget, clinic and personal-assistant environments. It also includes runnable merge-queue code, although those scenarios are outside that 74-scenario count. The repository provides evaluation and mitigation directories under an MIT license. Its availability was verified for this report; the experiments were not rerun. Code and documentation

The evidence has limits. These are simulations, and outcomes vary between runs. Claude models judged whether agents fabricated information or acted without consent, without validation against human judgments. The authors also describe substantial computational expense, which may constrain independent reproduction. Methods and limitations

For builders, the useful question is whether an agent system preserves each person’s constraints when several assistants change shared state. Benchmark scores alone cannot establish that a deployment does so. MAMUBench offers a concrete place to investigate that question.

Sources and citations

Revision history

Published .