Note · 2026-09-21

Agents that decide for themselves when to team up: two pilots

Last week we measured a team of agents with a fixed structure: a lead that splits the work three ways and three specialists that always investigate. It was accurate, and it cost five times the tokens of one trained agent. This time we removed the structure. There is no lead. Three agents each pick their own line of inquiry, keep a channel open, and team up only when they judge that joining forces would give a better answer than working alone. Whether they judge that well is something our environment can measure exactly, so we measured it, and then trained on it.

Two pilots, one after the other. In the first, all three agents could read all the data. Not one of 59 offers to team up that we could grade would have raised the score: the agents found the same causes, so there was never anything to combine. In the second, each agent held one department’s data, the way a real company splits it. They talked twice as much, and for the first time a merger occasionally helped. But on the cases built to need two departments, the trained team lost every one to a single agent that could see everything, trained on the same cases: 0.47 reward against 0.90 (p = 0.003). The single agent was the best result of this whole line of work.

These are pilots: one seed, 20 standard and 10 new held-out cases. Every investigation is also published agent by agent, so anyone can check what each agent did and said: every investigation, agent by agent.

Objective

The environment is the Business Investigation Arena from our first pilot: a simulated company whose revenue fell in one week, a database of its sales, orders, stock, shipments, customers and events, and a question: why? The grader knows the answer exactly, because the world is generated with the causes injected, so a conclusion is scored on arithmetic: the right causes, in the right scope, with the right share of the decline, and no decoys.

The question here comes straight from how people talk about agent teams: let the agents explore independently, give them a way to talk, and let them choose to cooperate when cooperation pays. We wanted to know three things. Do agents left to decide team up when it helps and not when it doesn’t? Does training on reward improve that judgement? And does the result beat one well-trained agent, which is the comparison that matters?

Plan

The protocol runs inside a single turn of the environment, so the tasks, the tools, the grader and the cost accounting are the same as for every other agent we have tested.

Three lanes, one per agent: each names an opening, then queries data in parallel, sends messages to the others that arrive at the start of their next turn, and may offer to team up; an accepted offer merges two lanes into one team with one shared draft. At the end, each team's draft is on the table and every agent endorses one; the most-endorsed draft is submitted. Below: after scoring, the verifier grades every offer's drafts apart and merged and the team without each agent, and a learner turns this into playbooks. — Figure 1. The protocol. Only one draft is submitted, so merging is the only way findings from different teams reach the same answer; it also imports the partner's mistakes.

Each agent starts as a team of one with its own draft conclusion. It can post a finding, offer another agent a merger, accept or decline an offer, or finish. Accepting merges the two teams and their drafts. At the end, every agent endorses one draft and the most-endorsed is submitted. Nothing in the instructions tells the agents when teaming pays: they are told the rules and how the case is scored, and the strategy is left to them.

The part we care most about is what happens after an investigation. The grader re-scores what was not submitted: every offer’s two drafts apart and merged, whether or not the merger happened, and the team’s answer without each agent’s findings and queries (that agent’s credit). It is the same trick the arena uses to measure each cause’s share, rebuilding the world without it, applied to the team. The agents never see any of it. A learner uses it to write two kinds of playbook: one about teaming, and one per agent about investigating. The model’s weights never change.

We fixed eight hypotheses in advance. The one that matters: a trained team matches or beats a single trained agent on reward.

Experiments

Model	`gpt-5.4`, frozen; learning is playbooks only
Standard cases	the same 20 held-out investigations as our earlier pilots (familiar causes, two causes at once, unseen cause types, a misleading decoy)
Cross-desk cases	10 new held-out investigations, each pairing a commercial cause (price increase, discount error, churn, partner loss, marketing cut, competitor) with an operational one (stock-out, warehouse outage), plus 12 for training
Pilot 1	three agents with the same data: untrained, and trained on standard cases
Pilot 2	three desks: commercial (customers, partners, marketing, prices, promotions), operations (orders, stock, suppliers, shipments, support) and platform (data loads, currency, the revenue decomposition), each able to read only its own tables plus the revenue data. The environment refuses anything else
Baselines	one agent trained on standard cases; one agent with all the data trained on the same cross-desk cases as the trained desk team
Training budget	12 cases, 2 steps of 12 investigations, identical for every trained arm
Cost	pilot 1: 15.7M tokens; pilot 2: 28.2M tokens

Outcomes

With the same data, there was nothing to combine

With all the data available to everyone, the untrained team was accurate: 80% of investigations fully correct, and not a single decoy claimed in 20 investigations. It also ran 62 queries per investigation. In most cases all three agents reached the same conclusion, usually by reading each other’s posted drafts, and each spent about 20 queries getting there. In a trial run on one two-cause case, the first agent found the full answer alone in 17 queries. That answer on its own would have scored 0.97; with the other agents’ queries on the bill, the team scored 0.79.

The offers to team up tell the story more directly.

One row of squares per team configuration. Same data, untrained: 6 offers, none raised the score. Same data, trained: 29 offers, none raised the score, 6 lowered it. Desks, untrained: 12 offers, 2 raised the score. Desks during the second training step: 14 offers, 4 raised the score by 0.37 to 0.41 each. Desks, trained: 1 offer in 30 investigations, which changed nothing. — Figure 2. Every offer to team up, graded apart and merged. With shared data, merging never helped. Split the data and it sometimes did, a lot, and then the trained team stopped offering.

Training made the team talk more, not better. Offers rose six-fold and mergers five-fold, mostly on single-cause cases where a merger cannot help, and the average merger was worth exactly 0.000. Queries fell by a quarter, but tokens rose 12%, because every call now carried the learned playbooks. On reward the trained team did not beat one trained agent (0.745 against 0.820, p = 0.44).

That pointed at the obvious gap in the experiment: teaming can only pay when agents know different things.

With split data, they talked but did not combine

Each agent now works one desk. The effect on behaviour was immediate: messages doubled (7.4 per investigation), each investigation had about two refused requests where one desk reached for another’s table, and the desks relayed findings to each other.

But a correct answer to a cross-desk case needs both desks’ causes in one draft, and the vote picks one draft. Here is what that looks like, from the recorded trace.

Timeline of one investigation with three lanes. Platform posts early that the drop is real and points to fulfilment; both other desks read it within seconds. Commercial has four requests refused because the decomposition belongs to platform, relays platform's finding to operations, and writes a draft with both causes, which would have scored 0.745. Operations posts its findings after both others have finished, so no one reads them, and drafts a single cause. All three vote for operations' draft, which scores 0.379. — Figure 3. One cross-desk investigation, agent by agent. The right answer was drafted; the vote submitted the other one.

This was not a one-off. On two-cause cases, half the causes the untrained desks missed (3 of 6) had been found by the desk that owned them and then lost at the vote. The rest were never found at all.

Training taught each desk to stay in its lane

Training on the cross-desk cases produced good lessons, desk by desk. Commercial learned to rule out price, promotion and marketing decoys first; operations to find the one warehouse behind a drop and measure the loss in the order statuses; platform to prove the drop is real, then hand the others a narrowed slice. The team as a whole got worse: 0.472 reward on cross-desk cases against 0.585 untrained, and lower on the standard cases too.

The teaming lessons explain why, and this is the most useful thing the pilot found. After the first training step the teaming playbook said it plainly:

If another peer’s evidenced cause is complementary to yours or your draft depends on it to be complete, offer a merge before finishing; do not rely on the final vote to combine separate drafts, because only one draft survives submission.

That is exactly right, and in the next training step, played under it, four mergers raised the score by 0.37 to 0.41 each. Then the final update replaced it with a stricter rule: absorb a partner’s finding “by message citation instead of offering a merge”, and merge only when the partner has posted a distinct cause with its family, scope, share and an explicit plan for any overlap. On the held-out cases the trained team made one offer in 30 investigations and merged none. The learner had found the right lesson and then edited it away.

What won: one agent, trained on the right cases

Reward by arm on 10 cross-desk and 20 standard held-out cases. One agent trained on two-cause cases: 0.895 and 0.900, 80% fully correct on both, about 80k tokens. One agent trained on standard cases: 0.663 and 0.820. Three agents with the same data, untrained: 0.705 and 0.722. Three desks, untrained: 0.585 and 0.746. Three desks, trained on two-cause cases: 0.472 and 0.641. — Figure 4. On the cases built for teams, the best team scored 0.705; one agent trained on the same kind of case scored 0.895, at a third of the tokens.

cases	comparison	reward difference, 95% CI	p	wins / losses
cross-desk	trained desks vs one agent trained on the same cases	−0.423 [−0.521, −0.309]	0.003	0 / 10
cross-desk	trained desks vs untrained desks	−0.113 [−0.230, −0.005]	0.14	6 / 4
cross-desk	one agent: trained on two-cause vs standard cases	+0.232 [+0.112, +0.343]	0.018	8 / 2
standard	trained desks vs one agent trained on the same cases	−0.260 [−0.405, −0.142]	< 0.001	0 / 20
standard	trained team, same data vs one agent trained on standard cases	−0.075 [−0.254, +0.120]	0.44	3 / 17

The same single agent, trained on two-cause cases instead of standard ones, went from 20% to 80% fully correct on cross-desk cases and scored 0.900 on the standard ones, the best of any arm we have measured in this environment. Its lessons are about attribution: measure each cause from its own mechanism rather than from the whole decline of the slice it sits in, and split overlapping causes explicitly. That is precisely the skill the desk teams lacked; their share estimates on cross-desk cases scored 0.47 to 0.54, against its 0.87.

Reading the record is how we found every error

The published trace shows each agent on its own lane: every query it ran, every request refused because the data belonged to another desk, every message from when it was sent to when each colleague actually read it, who was on which team when, and what each decision would have been worth. It is how we audited these pilots, and each of the following was found by reading it rather than by looking at scores:

The “shared” hypothesis log in the team we published last week was not shared: each specialist had a private copy. It affected one instruction. It is fixed.
The first credit rule credited an agent that copied another’s posted draft as if it had rediscovered the cause. Credit now stays with whoever found it.
A cause relayed in a message was credited to the agent that repeated it. Credit now goes to the agent that actually read the cause’s evidence.
The standard test set could not test split data: only 2 of its 20 cases needed two desks, and none of the training cases did. We caught this before running, which is why the cross-desk cases exist.
Found after publication (2026-09-23): documents were read only to their first line. The tools cut every text cell in a result to 77 characters, so 46% of the event rows these agents read lost their ending. For the injected causes that was mostly a date, which the sales data also shows; sometimes it was the deciding detail, and for a routine event, the words that cleared it (“no order impact recorded”). Every arm read the same cut text, so the comparisons stand. These pilots were not re-run; where we did re-run with full text, in our performance-review research, the headline comparisons held and each arm’s reward moved by less than run-to-run noise. Searches now show documents in full.

Conclusion

Teaming up pays only when agents know different things. Three copies of one model with the same data find the same answer, and a team of them is just the same investigation run three times. Give them different data and they do talk, but talking is not the hard part. Integration is: a team that ends by choosing one draft throws away whatever the minority found.

Training on reward did not fix that. A learner that compares better attempts with worse ones will improve whatever separates them, and here that meant sharper specialists and a worse team. It also found the right teaming rule and then edited it away. If team behaviour is the goal, the learner has to be shown whether the team combined its findings, not only whether it scored.

And the lever that moved the result most was not the team at all. One agent with all the data, trained on cases like the ones it would face, beat every team we built, on accuracy and on cost.

What this does not show yet:

These are pilots: one seed, one model, 10 cross-desk cases. The differences between the trained single agent and the teams are large and significant; most others are not.
The close is one design. A close that composes drafts (a union with overlaps resolved, or a lead whose only job is assembly) is untested, and it is the obvious next thing to try.
The intermediate teaming playbook, the one with the right rule, was never evaluated on held-out cases.
The learning here is playbooks on a frozen model. Weight updates might learn cooperation differently.

If this line continues, the next pilot keeps the desks and changes only the close: instead of voting on one draft, the team composes one from every desk’s findings. We would also give the learner an integration measure, whether each desk’s causes reached the submitted answer, and hold it to the counterfactual it is shown. For the job itself, the recommendation is already clear: train one agent on the cases it will face.

If you are building teams of agents over data that is split across departments, or deciding whether you should, we would like to hear from you.

All notes