BJBrian Jalaian
All news

Nov 2026 · AI Magazine · Vol. 47, Issue 4 · Special topic article · Open access

Safety in Numbers

Tackling the deliberative, performative, and infrastructural roots of multi-agent misalignment

Jeffrey M. Bradshaw (IHMC) and 22 co-authors, including Brian Jalaian

This AI Magazine article, led by Jeff Bradshaw at IHMC, brings together 23 authors from IHMC, the University of Bologna, Johns Hopkins APL, BYU, TNO, Ohio State and elsewhere. It asks what can go wrong when many AI agents, people and tools work together, and sketches a layered way to keep such teams safe. I am one of the co-authors.

Most work on AI alignment looks at one model at a time. When many agents work together, failures also come from how they coordinate and from the environment they run in. The article names three roots of multi-agent misalignment and proposes a three-layer safety pipeline to address them.

Three overlapping roots of multi-agent misalignment: deliberative (agents drift from shared commitments), performative (agents fail to coordinate or adapt), and infrastructural (the operating environment cannot be trusted or contained).Deliberativedrifting from shared commitmentscollusion · ignorance · disloyalty · deceptionPerformativefailing to coordinate or adaptpoor observability · competence limitsInfrastructuralan environment you cannot trustspoofing · leaks · cascading failuresCatastrophic failure ofthe multi-agent system
Three overlapping roots of multi-agent misalignment. A failure in one often feeds the others.

The problem

Multi-agent systems split complex work across many AI agents, people and tools. The agents may serve different principals, have different permissions, and cooperate on some goals while competing on others. That is powerful, but it also creates failure modes that no single-model safety test will catch.

The article groups these failures into three overlapping roots. Deliberative misalignment starts when agents drift from the commitments they share, through collusion, missing information, disloyalty or deception. Performative misalignment happens when well-intentioned agents still cannot coordinate, because they cannot see, predict or direct one another, or because a task is outside what they can do. Infrastructural misalignment comes from an operating environment that cannot be trusted: spoofed identities, leaked data, runaway resource use and failures that cascade from one agent to the next.

A layered safety pipeline

The authors propose one pipeline that addresses all three roots together, rather than in pieces.

  • Layer 1, before the work starts: every agent gets a clear identity, a record of who authorized it, and a smart teamwork contract that states the shared commitments and the consequences of breaking them.
  • Layer 2, while the agents work: oversight checks both goals and behavior, policy guardrails stop harmful actions before they happen, teamwork support helps agents coordinate, and a tamper-evident ledger records the evidence. Monitoring that evidence gives early warning of anomalies and cascading failures.
  • Layer 3, when something goes wrong: the evidence is analyzed to find what happened and who was responsible, and the contract's consequences are applied, or the contract is renegotiated.
The proposed three-layer safety pipeline: layer 1 establishes identity, authority and a smart teamwork contract; layer 2 oversees alignment, supports teamwork and gathers evidence while the agents work; layer 3 resolves disputes, applies consequences or renegotiates the contract.LAYER 1 · BEFORE THE WORKIdentity andauthority graphsSmart teamworkcontractcommitments + consequencesLAYER 2 · WHILE THE AGENTS WORKOversight of goals and behaviorinternal · goal alignment · policy guardrailsTeamworksupportEvidence: ledger andaccountability graphAnomaly detection and reputationall clearContinue the workbreachLAYER 3Resolve the disputeverify causesRenegotiatethe contractApply consequences
The three layers of the proposed pipeline. Most of the time the work simply continues; a dispute or breach sends it to layer 3.

Measuring teams, not just outcomes

The article also argues that a single outcome score says little about a team. Two robots may both finish a course in five minutes, yet one has half the top speed of the other, and the faster one made poor navigation choices along the way. The final time hides both facts.

Therefore, the authors propose measuring at three points: what the team could do by design, how well it adapts to surprises while it works, and how reliably it performs across many conditions afterwards.

Measuring a team at three points in time: design-time adaptability potential, run-time resilience as extensibility, and post-activity robustness. Outcome scores alone come only from the last stage.Design time: adaptability potentialoptions, capacity, competence envelopeRun time: resilience as extensibilityrecognize surprises, adjust roles and goalsPost-activity: robustnesssuccess, workload, time, across conditionsoutcome scores come only from the last box;the first two explain why the score came out that way
Design-time and run-time measures explain the post-activity score, instead of leaving it as a single number.

Why it matters to my research

My own work asks how to evaluate AI agents honestly, and how to tell when a system of agents is unsure or wrong. The article makes the same point at the scale of whole teams: an outcome score alone cannot tell you why a system succeeded or failed, and safety depends on noticing trouble early, from evidence gathered while the agents work.

The article is an initial sketch of a research agenda, not a results paper. The authors share it to invite others into the discussion.