Four identical agents and two messaging tools. After 115 steps of RL on chess puzzles, the team’s first-answer accuracy rises from 28% to 39%, close to its pass@4 of about 40%.
One puzzle. Four agents. Watch them check each other’s ideas.
26 s
Agent 4→ team
… I see 44...Qf4+ (f8f4) … Anyone see better, maybe a mate?
47 s
Agent 3→ team
Careful: f8f4 is illegal — the white pawn on f7 blocks the f-file …
82 s
Agent 2→ team
… My evaluation: best move is 44...Re7 (e1e7). … Anyone see something even stronger (forced mate)? …
105 s
Agent 4→ team
Agent 2's ...Re7 fails: 44...Re7 45.Qxe7Qxe7 46.Rxe7 and White wins a whole rook net … ...Re2+ remains best forcing move …
125 s
Agent 2→ team
You're right, I miscounted: … Re7 is refuted. I withdraw e1e7. … Going with e1e2.
129 s
Agent 4
The team’s first answer: …Re2+ ✓
All four agents ultimately submit the right move.
Most multi-agent systems are built with fixed structure: specialized subagents, an orchestration graph, or a parent agent that hands work to children [1, 2, 3]. We do the opposite and give the agents almost no structure [4] (see also concurrent work [5]). Four identical agents get the same task and two messaging tools, and RL trains them to work out for themselves how to communicate.
The task is chess puzzles. All four agents get the same puzzle, and the first answer submitted counts as the team’s answer. Before training, at least one of the four agents found the right move on 39% of held-out puzzles (pass@4), but the team’s first answer, which we treat as its pass@1, was right on only 28%. On 11% of puzzles, the first answer was wrong while a teammate had the right move.
We trained the team for 115 steps of RL with a rush penalty for submitting a wrong first answer when a teammate had the right one. First-answer accuracy rose from 28% to 39%, and pass@4 stayed between 38% and 42%. The agents are not solving more puzzles. Training converted most of pass@4 into pass@1, and the gap between them shrank from 11 points to 3. Along the way, agents learned to check with each other before submitting, and some teams began dividing up the work on their own.
Setup: four agents, one puzzle, and the first answer is scored
Each puzzle is a position from the Lichess puzzle set [6], rated between 2000 and 2800, and the task is to name the first move of the solution, which usually means calculating the whole line. All four agents are the same model with the same prompt. They work at the same time and have two tools:
send_message(to, content) sends content, the message text, to the teammates in to: either "all" or a list of agent numbers such as ["2", "3"]. Each recipient sees the message the next time it calls a tool.
check_inbox(wait_seconds) returns any unread messages. If there are none, it waits up to wait_seconds and returns as soon as a message arrives.
An agent submits by ending a turn without a tool call, and its answer is the move inside \boxed{} in that turn. Submitting is final. The agent stops, and messages sent to it afterwards are never read.
One puzzle
Agent 1
Agent 2
Agent 3
Agent 4
can all message each other
answers first
First answer submitted= the team’s answer
Every agent gets the same puzzle and submits its own answer. Only the first one submitted is scored as the team’s answer. Here Agent 2 answers first.
We evaluate the team on its first submitted answer, because that gives exactly one answer per puzzle, which is what a user of the team would get. Other common metrics do not isolate a single submission. pass@k and mean@k score each agent’s answer separately and never say what the team as a whole answered. maj@k (majority vote) picks one answer only when there is a clear majority, so it is unclear what the team submits when the votes tie. It also needs answers that can be matched as identical, which fails for open-ended outputs such as math proofs.
We still report the team’s pass@4 as an upper bound: if any of the four agents found the right move, the team could have submitted it. This pass@4 is measured within a single team conversation, where the agents read each other’s messages, so their four answers are not independent samples like those in the usual pass@k. We aim to turn the team’s pass@4 into first-answer accuracy.
Before training, teams often had the right answer but submitted incorrectly
The puzzle below is a rook endgame with one pawn each, White to move. Black’s rook attacks the pawn on d5. The winning move is to push the pawn to d6, where White’s rook on a6 protects it along the sixth rank. The other candidate is a check with the rook on a8. That moves the rook off the sixth rank, the pawn loses its protection, and the position is a draw. This board replays the team’s submitted Ra8+ line, and the bar beside it shows the engine’s evaluation.
Team’s submitted moveRa8+
8
7
6
5
4
3
2
1a
b
c
d
e
f
g
h
♚︎♜︎♜︎♟︎♟︎♚︎
Chess position, White at the bottom. Black king f8, Black rook d7, White rook a6, Black pawn h6, White pawn d5, White king f5.
Before training
27 s
Agent 2
Key observation: d5 pawn is attacked by Rd7, and 61.d6! advances it with protection — the rook on a6 guards d6 along the 6th rank … Anyone see a better/forcing move (e.g. Ra8+)?
Full message from Agent 2 at 27 s
Position: White Ra6, Pd5, Kf5; Black Kf8, Rd7, Ph6. White to move.
Key observation: d5 pawn is attacked by Rd7, and 61.d6! advances it with protection — the rook on a6 guards d6 along the 6th rank (a6-b6-c6-d6 clear). If 61...Rxd6?? then 62.Rxd6 wins the rook. The passed pawn becomes decisive.
Anyone see a better/forcing move (e.g. Ra8+)?
29 s
Agent 3
Key idea: 1.d6! pushes the passed pawn and crucially the pawn on d6 is protected by the rook on a6 along the 6th rank … Alternative Ra8+ is also forcing but let me check what you others think.
Full message from Agent 3 at 29 s
Analyzing: White Ra6, Kf5, Pd5 vs Rd7, Kf8, Ph6. Black threatens Rxd5 (d5 is only defended... nothing defends it actually). Key idea: 1.d6! pushes the passed pawn and crucially the pawn on d6 is protected by the rook on a6 along the 6th rank, so ...Rxd6 loses to Rxd6+. Seems strong. Alternative Ra8+ is also forcing but let me check what you others think.
34 s
Agent 1
I’m leaning 1.d6: pawn advances to a protected square … Anyone see a tactical refutation or a stronger move (e.g. Ra8+)?
Full message from Agent 1 at 34 s
Position: White Kf5 Ra6 Pd5; Black Kf8 Rd7 Ph6; White to move. I’m leaning 1.d6: pawn advances to a protected square (Ra6 guards d6 along the 6th rank), and after d6 the e7 square is controlled so Black can't easily bring the king; if Rxd6 then Rxd6 wins the rook. Rxh6 allows Rxd5 and likely just a drawn rook ending. Anyone see a tactical refutation or a stronger move (e.g. Ra8+)?
39 s
Agent 4The team’s answer: Ra8+ ✗
Agent 4 had sent no message and had not read the three above.
42 s
Agent 1Answers: d6 ✓
42 s
Agent 3Answers: d6 ✓
43 s
Agent 2Answers: d6 ✓
Puzzle PjW4K, rated 2737, held-out set, step 0. Messages are quoted from the log, with cuts marked by an ellipsis. Times are seconds from the start. The evaluation bar is Stockfish 19 at depth 22.
Three agents found d6 and sent it to the team. Two of them asked whether Ra8+ was better. Agent 4 answered Ra8+ at 39 seconds. It had sent no message and had not read the three addressed to it. Ra8+ became the team’s answer.
This happened often. On the held-out set before training, the first answer was wrong while a teammate had the right move on 11% of puzzles, and 40% of agents never sent a message.
Reward design
We believe collaboration is key to first-answer accuracy. However, we avoided directly rewarding messaging because agents could learn to send messages only to collect the reward without helping the team get the first answer right, a form of reward hacking. Instead, we designed the reward around three desirable properties:
Prioritize first-answer accuracy. The team is evaluated on its first submitted answer, so that answer’s correctness carries most of the reward.
Keep every agent working toward a correct solution. All agents should have an incentive to eventually find the right answer, even if they submit later.
Use what the other agents know. The team’s answer should incorporate information from multiple agents, so collaboration can improve on the submitting agent’s independent reasoning.
Initial reward attempt
Our first formulation aimed to satisfy all three properties, with no explicit optimization pressure to communicate before submitting a response. It rewarded the team for a correct first answer and each agent for its own correct answer.
rk=0.8 Tteam’s first answer correct+0.2 Ckown answer correct
(1)
Here rk is the reward for agent k. T = 1 if the first submitted answer is correct, and 0 otherwise. Every agent receives this shared reward, so helping whoever submits first get it right benefits everyone. Ck = 1 if agent k’s own final answer is correct, and 0 otherwise. Ck gives every agent an incentive to solve the puzzle itself.
With this reward, messaging did not change: over 57 steps, about 36% of agents still never sent a message. Losing the shared 0.8 affects everyone equally; it gives no extra penalty to the agent whose wrong submission became the team’s answer.
Initial reward (no rush penalty)
Agents that never send a message
Share of agents per training batch
0%
10%
20%
30%
40%
50%
36%37%
0204060
RL training step
First answers before messaging
Share of first answers from an agent that had neither sent nor read a message
0%
10%
20%
30%
40%
50%
60%
33%37%
0204060
RL training step
Initial reward, steps 0–57. Faint lines show individual training steps; bold lines show a 10-step moving average.
Rush penalty
To more closely address the third property and encourage communication, we introduced a rush penalty for submitting a wrong first answer when a teammate went on to find the right one. Rushing to a wrong answer now costs the first submitter extra, so it pays to check with the team before committing.
rk=0.8 Tteam’s first answer correct+0.2 Ckown answer correct−0.2 Rkrushed a wrong answer
(2)
Rk = 1 only if agent k submits first, its answer is wrong, and at least one teammate’s final answer is correct. Otherwise Rk = 0. Only the first submitter receives this penalty.
Why the new penalty helps
In the rook endgame from the previous section, three agents found the winning move, d6, and sent it to the team. Agent 4 did not read those messages and submitted Ra8+ first. Its wrong answer became the team’s answer, even though the team already had the solution.
Initial reward: Agent 4 gets 0. The team’s first answer is wrong (T = 0), and Agent 4’s own answer is wrong (C4 = 0). It receives 0.8 × 0 + 0.2 × 0 = 0. There is no additional cost for submitting before using its teammates’ correct solution.
With the rush penalty: the same decision earns −0.2. Agent 4 submitted first, was wrong, and had a correct teammate, so R4 = 1. Its reward becomes 0 − 0.2 = −0.2. The three correct teammates still receive 0.2 each; the extra penalty falls specifically on Agent 4.
The new term assigns an extra cost to turning a solvable puzzle into a wrong first answer. This is intended to encourage agents to check and use their teammates’ reasoning before submitting. The reward still comes from the answers, and sending more messages alone earns nothing.
Initial reward (no rush penalty)With the rush penalty
Agents that never send a message
Share of agents per training batch
10%
20%
30%
40%
36%37%13%38%
020406080100115
RL training step
First answers before messaging
Share of first answers from an agent that had neither sent nor read a message
0%
10%
20%
30%
40%
50%
33%37%2%47%
020406080100115
RL training step
The orange run ends at step 57; the blue run continues through step 115 in both plots. Faint lines show individual training steps; bold lines show a 10-step moving average.
Results: higher first-answer accuracy
On 100 held-out puzzles, first-answer accuracy rose from 28% to 39%. pass@4 stayed between 38% and 42%, so the share of puzzles that at least one agent solved barely changed. The team got better at submitting the right answer when one of its agents had it. The gap between pass@4 and first-answer accuracy shrank from 11 points to 3.
First-answer accuracy and pass@4 on held-out puzzles
Share of puzzles, by training checkpoint
At least one agent correct (pass@4)Team's first answer correct
25%
30%
35%
40%
45%
42%39%39%28%
020406080100115
RL training step
100 held-out puzzles per checkpoint, so each point is uncertain by about ±5 points.
Messaging during training
As training progressed, agents sent more messages and checked their inbox more. Messages sent per agent rose from 1.3 to 2.3, inbox checks from 1.4 to 2.2, and turns from 3.5 to 5.1.
Communication per agent
Messages sentInbox checks
1.0
1.5
2.0
2.5
2.21.42.31.3
020406080100115
RL training step
Training batches from the rush-penalty run, steps 0 to 115. Faint lines are single steps; bold lines are a 10-step moving average.
Emergent behavior: agents organize their own work
All four agents get the same prompt and the same messaging tools. We do not assign a leader, roles, or a division of work, and the reward depends only on correct answers, not on messaging or delegating. However, late in training we observed agents organizing their own work. In the two conversations below, one agent took a coordinating role and assigned candidate moves to its teammates, and the other agents reported back on their assigned moves.
Dividing candidate moves
One agent assigns a different king move to each teammate; all three report back on the candidate lines.
90 s
Agent 4
Let's divide candidates. Agent1: analyze 33.Kd3 (likely best) and black replies.Agent2: analyze 33.Ke3 and refutes if losing.Agent3: analyze 33.Kc4. …
Full message from Agent 4 at 90 s
Let's divide candidates. Agent1: analyze 33.Kd3 (likely best) and black replies. Agent2: analyze 33.Ke3 and refutes if losing. Agent3: analyze 33.Kc4. Use endgame tree; report principal lines.
126 s
Agent 2
Agent 2 update on 33.Ke3: black has an immediate 33...f4+ resource. … Waiting on Agent1 (Kd3) and Agent3 (Kc4) lines.
Full message from Agent 2 at 126 s
Agent 2 update on 33.Ke3: black has an immediate 33...f4+ resource. If 34.gxf4 gxf4+ 35.Kxf4 (if legal) White seems to trade g-pawn for f/g pawns and may come out a pawn up with active king (white a4 c3 f3 h2 vs black a5 c5 h6) — need verify legality of Kxf4 and black king distance. Alternatively black can decline with Kc6. I don't think Ke3 is refuted yet. Waiting on Agent1 (Kd3) and Agent3 (Kc4) lines.
214 s
Agent 3
Agent 3 analysis of 1.Kc4: after 1...f4 2.gxf4 gxf4, the black f4-pawn is blocked by White's f3-pawn, so it is not a runner. …
Full message from Agent 3 at 214 s
Agent 3 analysis of 1.Kc4: after 1...f4 2.gxf4 gxf4, the black f4-pawn is blocked by White's f3-pawn, so it is not a runner. White can play 3.Kb5 and (since ...Kc6 would be adjacent to the b5 king) Black cannot easily defend both a5 and c5; White wins at least a5 or c5. Against immediate 1...Kc6, 2.Kb5 Kb7/c7 Kd6 etc., White wins a5 or c5 (Kxc5 is legal if the king no longer protects it; Kxa5 otherwise). This looks strong. Is there a black tactical refutation before Kb5?
400 s
Agent 1
Decisive point: after Kd3/Ke3, ...Kd5! centralizes … while still guarding c5. …
Full message from Agent 1 at 400 s
No concrete refutation found for d4c4. Decisive point: after Kd3/Ke3, ...Kd5! centralizes (f3 only hits e4/g4, c3 hits d4/b4, white king no longer controls d5) while still guarding c5. After Kc4, ...Kd5 is illegal (white king controls d5); Black must keep c5 guarded. Pawn moves such as ...f4 are met by gxf4 gxf4 Kb5!, and ...Kc6 prevents Kb5 but leaves black passive; ...Kd6/...d5 etc. lose c5 or a5. Kc4 is the best evasion. Support d4c4; speak up only if you have a concrete tactic.
Full conversation · 16 messages
37 s
Agent 3
Position after ...c5+: White Kd4, pawns a4 c3 f3 g3 h2; Black Kd6, pawns a5 c5 f5 g5 h6. Legal check responses include Kc4, Kd3, Ke3 (Kxc5 illegal due to Kd6; Ke4 illegal due to f5 pawn). I think Kc4 is the active move. Is anyone else checking the endgame?
37 s
Agent 1
Position FEN: 8/8/3k3p/p1p2pp1/P2K4/2P2PP1/7P/8 w - - 0 33. White in check from c5 pawn. Legal king moves seem Kc4, Kd3, Ke3 (maybe). Need find best pawn-endgame move. I'll analyze; share any findings.
49 s
Agent 4
Position: White K d4, pawns a4 c3 f3 g3 h2; Black K d6, pawns a5 c5 f5 g5 h6. Black just played c6-c5+ (pawn on c5 gives check to K d4). Legal responses seem only Kc4, Kd3, Ke3 (Kxc5 and Kd5/e5 illegal due to black king; e4 attacked by f5 pawn; c3 occupied). Need find best move. I'm leaning Kd3/Ke3 depending on opposition; please share analysis.
90 s
Agent 4
Let's divide candidates. Agent1: analyze 33.Kd3 (likely best) and black replies.Agent2: analyze 33.Ke3 and refutes if losing.Agent3: analyze 33.Kc4. Use endgame tree; report principal lines.
96 s
Agent 2
I’ve mapped the position: White Kd4, pawns a4 c3 f3 g3 h2; Black Kd6, pawns a5 c5 f5 g5 h6. Black’s c5+ checks along the pawn attack on d4. Legal White replies appear to be only Kc4, Kd3, Ke3 (Kc5/Kd5/e5 illegal due to black king, e4 due to f5 pawn). I suspect Kc4 is best (attacks/clamps c5) but I’m not certain; please share your evaluations.
126 s
Agent 2
Agent 2 update on 33.Ke3: black has an immediate 33...f4+ resource. If 34.gxf4 gxf4+ 35.Kxf4 (if legal) White seems to trade g-pawn for f/g pawns and may come out a pawn up with active king (white a4 c3 f3 h2 vs black a5 c5 h6) — need verify legality of Kxf4 and black king distance. Alternatively black can decline with Kc6. I don't think Ke3 is refuted yet. Waiting on Agent1 (Kd3) and Agent3 (Kc4) lines.
178 s
Agent 3
My current conclusion: 1.Kc4 is too slow because Black can play ...f4, forcing gxf4 gxf4 and create a dangerous f-pawn (or after a queenless race White can't stop it). 1.Ke3 both evades check and keeps the king close enough to meet ...f4+ with gxf4 gxf4 Kxf4, while remaining centralized. Kd3 is a more passive version. I lean d4e3.
187 s
Agent 1
I’m converging on 1.Kc4 (UCI d4c4): Kxc5 illegal (Kd6), Ke4 illegal (f5), Kd3/Ke3 allow ...Kd5 and lose activity. Kc4 forces ...Kc6 and keeps white king active attacking c5; black must respond. Any objection or alternative?
208 s
Agent 4
Agent2: note after 34.gxf4 black g5xf4+ not ...gxf4+? same thing; then 35.Kxf4 and white is up a pawn but check black king d6/e5 maybe can attack? Need full line. Agent1/Agent3: please try to prove best among Kd3/Kc4/Ke3. Also consider after 33.Kd3 possible 33...Kd5 34.c4+!? and after 33.Kc4 33...Kc6 opposition.
214 s
Agent 3
Agent 3 analysis of 1.Kc4: after 1...f4 2.gxf4 gxf4, the black f4-pawn is blocked by White's f3-pawn, so it is not a runner. White can play 3.Kb5 and (since ...Kc6 would be adjacent to the b5 king) Black cannot easily defend both a5 and c5; White wins at least a5 or c5. Against immediate 1...Kc6, 2.Kb5 Kb7/c7 Kd6 etc., White wins a5 or c5 (Kxc5 is legal if the king no longer protects it; Kxa5 otherwise). This looks strong. Is there a black tactical refutation before Kb5?
236 s
Agent 3
Unless anyone sees a concrete refutation, I plan to give d4c4. It is the only check evasion that seizes queenside space and prevents ...Kd5 while keeping ...f4 under control.
247 s
Agent 2
Key point: after 33.Kd3 or 33.Ke3, Black can play ...Kd5! (d5 is not attacked; White f3 only hits e4/g4, c3 hits d4). That centralizes Black's king while still defending c5. After 33.Kc4, ...Kd5 is illegal because the White king on c4 controls d5, and Black must keep Kd6/Kc6 to guard c5. So Kc4 is the only check evasion that doesn't concede the d5-square. Agent1's d4c4 looks right to me unless someone has a concrete refutation of Kc4.
249 s
Agent 3The team’s answer: Kc4 ✓
259 s
Agent 2
I agree with d4c4 too. One caveat to Agent3's line: after 33.Kc4 Kc6, 34.Kb5 is illegal because b5 is adjacent to the black king on c6. But the main point stands: Kc4 prevents ...Kd5, keeps the white king active, and if Black spends a move on ...f4 while Kd6 remains, then after gxf4 gxf4 the f4-pawn is blocked by f3 and White can consider Kb5 winning a5. Unless Agent4 has a concrete refutation, I'll support d4c4.
329 s
Agent 4
Need resolve. Key lines: (A) 33.Kc4 Kc6 34.Kd3 Kd6 35.c4 — assess. (B) 33.Kd3 Kd5 34.c4+ Ke5 — is this bad for White? Also 33.Kd3 avoids ...f4 with check? Please be concrete; avoid assuming.
355 s
Agent 2
Concrete assessment: In (B), 33.Kd3 Kd5 34.c4+ does NOT win because Kxc4 is illegal due to White Kd3, but Black has 34...Ke5! (or Ke6) with a very active king; e5 is not attacked by Kd3/f3/c4. White then has no useful c-pawn break and the c4-pawn may become weak. That's why Kd3/Ke3 concede d5. In (A), White isn't forced into 34.Kd3; after 33.Kc4 Kc6 White can make a kingside waiting move (e.g. h3/h4) and Black's king must remain tied to c5/a5. If Black instead leaves d6 while c5/a5 are exposed, White wins a5 or c5. So Kc4 remains my choice; no refutation shown.
400 s
Agent 1
No concrete refutation found for d4c4. Decisive point: after Kd3/Ke3, ...Kd5! centralizes (f3 only hits e4/g4, c3 hits d4/b4, white king no longer controls d5) while still guarding c5. After Kc4, ...Kd5 is illegal (white king controls d5); Black must keep c5 guarded. Pawn moves such as ...f4 are met by gxf4 gxf4 Kb5!, and ...Kc6 prevents Kb5 but leaves black passive; ...Kd6/...d5 etc. lose c5 or a5. Kc4 is the best evasion. Support d4c4; speak up only if you have a concrete tactic.
400 s
Agent 4Answers: Kc4 ✓
402 s
Agent 2Answers: Kc4 ✓
411 s
Agent 1Answers: Kc4 ✓
All four agents eventually submitted the right move.
Puzzle PN9ax
Reporting back to a coordinator
An agent offers to synthesize the analysis while teammates check separate forcing lines.
202 s
Agent 1
Let's divide the forcing lines. … Agent 2: analyze 1...Qf4+ and 1...Qe5+ concretely: … Agent 3: analyze 1...Rg1+ concretely: … Agent 4: analyze 1...Qh4+ and 1...Qg4+ and Qxf3+: confirm refutations and material outcome; … I will synthesize. …
Full message from Agent 1 at 202 s
Let's divide the forcing lines. Agent 2: analyze 1...Qf4+ and 1...Qe5+ concretely: list ALL legal White responses to each, including Rxf4/Rf4 blocks and king moves, and evaluate outcome. Agent 3: analyze 1...Rg1+ concretely: White responses (Rg2 block, Kh3, etc.) and the follow-up; find if it forces mate/win. Agent 4: analyze 1...Qh4+ and 1...Qg4+ and Qxf3+: confirm refutations and material outcome; also check if any quiet move is stronger. I will synthesize. Please report concrete refutations or forced mates, not generalities.
277 s
Agent 4
Agent 4 report: ...Qh4+ fails to 30.Qxh4 (h8-h4 clear); … ...Qg4+ fails to Kxg4 (g4 unprotected). …
Full message from Agent 4 at 277 s
Agent 4 report: ...Qh4+ fails to 30.Qxh4 (h8-h4 clear); after ...gxh4+ Kxh4 Black has only R+N+4P vs 2R+N+4P, bad. ...Qg4+ fails to Kxg4 (g4 unprotected). ...Qxf3+ fails to Kxf3 (f3 unprotected; even ...Rd3+ is only a check, down queen). Also note Black has only 5 pawns (a7 c6 f6 b5 g5). For ...Rg1+: 30.Kh3 Qg4# works (Qg4 protected by Rg1 on g-file). If 30.Rg2 (Rf2-g2), 30...Rxg2+ 31.Kxg2 is legal because f3 still blocks Qe4-g2; that trades rooks, not mate. Need a stronger 30th move after Rg2 or accept it's still best/forcing. Can anyone find 30...? after Rg2 (e.g., quiet move exploiting pin) that wins/mates?
374 s
Agent 2
Agent 1 asked me to detail Qf4+/Qe5+. I confirm: … 1...Qf4+ 2.Rxf4 is legal (the checker is captured with a rook, despite g5 pawn guarding f4), so not a mate. …
Full message from Agent 2 at 374 s
Agent 1 asked me to detail Qf4+/Qe5+. I confirm:
1...Qf4+ 2.Rxf4 is legal (the checker is captured with a rook, despite g5 pawn guarding f4), so not a mate.
1...Qe5+ is the most plausible alternative. After 2.Rf4 gxf4+, Kxf4 is illegal because Qe5 guards f4, but White can move (including Kf3/g4/h3/h4/f2 perhaps) or R with remaining/pinned? Need determine best.
I still think 1...Rg1+ is intended. Need find strongest continuation after 2.Rg2: 2...Rxg2+, 2...Qe1+, or 2...Qe5+. Which wins? Please focus if calculating.
428 s
Agent 2
My concrete results: ...Qg4+ Kxg4; ...Qxf3+ Kxf3; ...Qh4+ Qxh4; ...Qf4+ Rxf4. … Unless someone finds a specific mate, I vote d1g1. Any objections/corrections?
Full message from Agent 2 at 428 s
My concrete results: ...Qg4+ Kxg4; ...Qxf3+ Kxf3; ...Qh4+ Qxh4; ...Qf4+ Rxf4. 1...Rg1+ gives Kh3 then Qg4#, or forced Rg2 and Black can Rxg2+, equalizing the extra rook White has. 1...Qe5+ wins the exchange maybe but permits multiple king moves and no forced mate. Unless someone finds a specific mate, I vote d1g1. Any objections/corrections?
Full conversation · 16 messages
47 s
Agent 2
I'm mapping Black's tactical checks after White Kg3. Candidate queen checks are Qxf3+, Qf4+, Qh4+, Qg4+. Need verify defenses (especially Rxf4 and K escapes) and see if there's a mate. What have you found?
69 s
Agent 4
I’m mapping White: Kg3 Qh8 Rf3 Rf2 Na3 pawns a2 b3 c3 h2; Black: Kf8 Ng8 Qe4 Rd1 pawns a7 c6 f6 b5 g5. Black to move. Candidate checks: ...Qf4+ (met by Rxf4?), ...Qh4+ (met by Qxh4), ...Rg1+ (maybe strong). Looking for mate/best move. What are you finding?
111 s
Agent 1
I'm analyzing the tactic. Key board: White Kg3, Qh8, rooks f3/f2, knight a3, pawns a2 b3 c3 h2. Black Qe4, Rd1, Kg8? actually Kf8, Ng8, pawns a7 c6 f6 b5 g5. Candidate checks: Qf4+ (protected by g5, but Rxf4 possible), Qh4+ (protected by g5, but Qxh4 possible), Qe5+ (f4 block possible), Rg1+ maybe. Need find best/forced mate. What are you seeing?
152 s
Agent 3
I'm analyzing. Initial candidates: ...Qh4+ forces Kg2? ...Qf4+ can be Rxf4. ...Rg1+ has Kh3/Rg2. Need identify intended best move/mate. Any ideas?
163 s
Agent 4
Key point: White Qh8 defends h4 down the h-file (h7/h6/h5 clear), so ...Qh4+ is met by Qxh4. ...Qf4+ is met by Rxf4. ...Qg4+ may be met by Kxg4. The promising checking move seems ...Rg1+ (line on g-file), but Kh3 or Rf2-g2 need analysis. Also wondering if there's a forced mate after ...Rg1+ Kh3 or if best is something else. Please verify h-file/queen defense and Rg1 lines.
173 s
Agent 2
Key line found: 1...Rg1+! checks down the g-file. If 2.Kh3, then 2...Qg4 is mate (queen protected by Rg1). If 2.Rg2, need determine Black's best continuation/evaluation. Does anyone have engine-style confirmation of 1...Rg1+ as best, or another first move?
181 s
Agent 3
Important: ...Qh4+ is met by Qxh4 down h-file (h7-h5 clear). ...Qf4+ met by Rxf4. Need consider ...Qe5+ (check along e5-f4-g3, queen protected by f6). Blocks: Rf4 then gxf4+ maybe. King moves h3/g2? Need check attacks. Does anyone see mate after Qe5+?
194 s
Agent 2
After 1...Rg1+ white has essentially 2.Kh3 (then 2...Qg4#, rook protects queen) or 2.Rg2 (block; Black needs continuation). Are all three seeing that Qh4+ fails to Qxh4 and Qf4+ fails to Rxf4? If anyone has a forced mate/evaluation after 2.Rg2, share it.
202 s
Agent 1
Let's divide the forcing lines. Agent 2: analyze 1...Qf4+ and 1...Qe5+ concretely: list ALL legal White responses to each, including Rxf4/Rf4 blocks and king moves, and evaluate outcome. Agent 3: analyze 1...Rg1+ concretely: White responses (Rg2 block, Kh3, etc.) and the follow-up; find if it forces mate/win. Agent 4: analyze 1...Qh4+ and 1...Qg4+ and Qxf3+: confirm refutations and material outcome; also check if any quiet move is stronger. I will synthesize. Please report concrete refutations or forced mates, not generalities.
226 s
Agent 4
I found another check: ...Qe5+ (diagonal e5-f4-g3). But it seems to allow Kg4/Kh3/Kg2 and the f4 block Rf3-f4. We need be rigorous. Let's list king escape squares for each candidate and what controls them. I also think h3 is often the problem escape; if f3 rook were removed, Qe4 would control g2 but not h3. Maybe the intended move involves ...Rg1+ to control g2 and ...? Please share exact forced mate if any.
247 s
Agent 2
Concrete check refutations:
- 1...Qf4+ 2.Rxf4 gxf4+ 3.Rxf4 (or even Kxf4 illegal because Qe5? correction, queen was captured) Black loses queen; note after 2...gxf4+ pawn protected only by... queen is gone, but f4 pawn also protected by? Qe5 gone; knight? no. White can likely Rxf4. Definitely not forced mate.
- 1...Qh4+ 2.Qxh4 gxh4+ 3.Kxh4/Kg? queens off and White remains better.
- 1...Qg4+ 2.Kxg4.
- 1...Qxf3+ 2.Kxf3.
- 1...Qe5+ seems much more dangerous: 2.Rf4 gxf4+; Kxf4 is illegal due Qe5 protecting f4. Need see White's best reply to this pawn check (3.Rxf4 Qxf4+ 4.Kxf4 etc; or king moves) and king escapes instead of block.
- 1...Rg1+ as previously.
Which is objectively best? Need decide based on calculation.
277 s
Agent 4
Agent 4 report: ...Qh4+ fails to 30.Qxh4 (h8-h4 clear); after ...gxh4+ Kxh4 Black has only R+N+4P vs 2R+N+4P, bad. ...Qg4+ fails to Kxg4 (g4 unprotected). ...Qxf3+ fails to Kxf3 (f3 unprotected; even ...Rd3+ is only a check, down queen). Also note Black has only 5 pawns (a7 c6 f6 b5 g5). For ...Rg1+: 30.Kh3 Qg4# works (Qg4 protected by Rg1 on g-file). If 30.Rg2 (Rf2-g2), 30...Rxg2+ 31.Kxg2 is legal because f3 still blocks Qe4-g2; that trades rooks, not mate. Need a stronger 30th move after Rg2 or accept it's still best/forcing. Can anyone find 30...? after Rg2 (e.g., quiet move exploiting pin) that wins/mates?
364 s
Agent 4
I confirm with Agent 4 refutations. I’m leaning ...Rg1+ as best: if Kh3 then Qg4#, and if Rf2-g2 then Rxg2+ at least trades rooks with the f3 rook becoming pinned to Kg2 after Kxg2 (though no immediate mate). Qe5+ allows Kg4/Kg2/Kh3 and Rf4 block, so less forcing. Does anyone see a mate or clear win after Rg2 that makes ...Rg1+ unquestionably best, or an alternative first move stronger than ...Rg1+?
374 s
Agent 2
Agent 1 asked me to detail Qf4+/Qe5+. I confirm:
1...Qf4+ 2.Rxf4 is legal (the checker is captured with a rook, despite g5 pawn guarding f4), so not a mate.
1...Qe5+ is the most plausible alternative. After 2.Rf4 gxf4+, Kxf4 is illegal because Qe5 guards f4, but White can move (including Kf3/g4/h3/h4/f2 perhaps) or R with remaining/pinned? Need determine best.
I still think 1...Rg1+ is intended. Need find strongest continuation after 2.Rg2: 2...Rxg2+, 2...Qe1+, or 2...Qe5+. Which wins? Please focus if calculating.
428 s
Agent 2
My concrete results: ...Qg4+ Kxg4; ...Qxf3+ Kxf3; ...Qh4+ Qxh4; ...Qf4+ Rxf4. 1...Rg1+ gives Kh3 then Qg4#, or forced Rg2 and Black can Rxg2+, equalizing the extra rook White has. 1...Qe5+ wins the exchange maybe but permits multiple king moves and no forced mate. Unless someone finds a specific mate, I vote d1g1. Any objections/corrections?
464 s
Agent 2The team’s answer: Rg1+ ✓
555 s
Agent 4
I concur: ...Rg1+ is the best move. It forces either Kh3 (then ...Qg4# because Rg1 protects g4 down the g-file) or Rf2-g2. After Rg2, ...Rxg2+ Kxg2 leaves Rf3 pinned on e4-f3-g2 and ...g4! attacks it twice; Black wins material/keeps a decisive attack. Other checks fail: ...Qh4+ Qxh4, ...Qf4+ Rxf4, ...Qg4+ Kxg4, ...Qxf3+ Kxf3; ...Qe5+ allows escapes/block. I’m ready to answer d1g1 unless Agent 1/3 object.
558 s
Agent 4Answers: Rg1+ ✓
Two agents submitted the right move; the other two finished without an answer.
Puzzle CznY2
Four selected messages per conversation, in chronological order; … marks cuts. Assignment colors match the corresponding agent labels. Expand any message to read the full text, or open “Full conversation” to read every message and answer in the session. There, yellow marks where agents report back on an assigned line, correct a teammate, or ask for objections before answering.
The same puzzle after training
The rook endgame from above, at the step-80 checkpoint:
Team’s submitted moved6
8
7
6
5
4
3
2
1a
b
c
d
e
f
g
h
♚︎♜︎♜︎♟︎♟︎♚︎
Chess position, White at the bottom. Black king f8, Black rook d7, White rook a6, Black pawn h6, White pawn d5, White king f5.
After training
31 s
Agent 3
Idea 1.d6 uses Ra6’s rank defense of d6, so Rxd6 loses to Rxd6. Also candidate 1.Ra8+ gives check … Need compare. What are you finding?
Full message from Agent 3 at 31 s
I see White: Ra6, Kf5, Pd5 vs Black: Kf8, Rd7, h6. Black's rook attacks d5 down the file (d7-d6-d5). Idea 1.d6 uses Ra6's rank defense of d6, so Rxd6 loses to Rxd6. Also candidate 1.Ra8+ gives check; if Rd8 block then Rxd8+ wins exchange. Need compare. What are you finding?
37 s
Agent 1
My candidate is d5d6 … Checking others’ candidates before final.
Full message from Agent 1 at 37 s
Position: White Kf5, Ra6, pawn d5 vs Black Kf8, Rd7, pawn h6. My candidate is d5d6: the pawn is taboo because Rxa? actually ...Rxd6 is met by Rxd6 from a6 along the 6th rank. It creates a protected passed pawn and dislodges/blocks the rook. Checking others' candidates before final.
45 s
Agent 4
Idea: 1.d6! — pawn advances, and Rxd6?? loses to Rxd6 … This looks like the winning move. Checking with you.
Full message from Agent 4 at 45 s
Position: White Ra6, Pd5, Kf5 vs Black Kf8, Rd7, Ph6. Idea: 1.d6! — pawn advances, and Rxd6?? loses to Rxd6 (rook a6 sees d6 along the 6th rank). Black can't Ke7 since pawn on d6 controls e7. This looks like the winning move. Checking with you.
49 s
Agent 2
I’m leaning toward 61.Ra8+ (a6a8) since it forces Rd8 block … or Ke7 (then d6+ with Ra6 guarding d6). Checking whether anyone sees a mate or stronger.
Full message from Agent 2 at 49 s
Position: White Ra6, Kf5, Pd5 vs Black Kf8, Rd7, Ph6. White to move. I'm leaning toward 61.Ra8+ (a6a8) since it forces Rd8 block (Rxd8+ and d6 wins) or Ke7 (then d6+ with Ra6 guarding d6). Checking whether anyone sees a mate or stronger.
50 s
Agent 4
Agreeing on d5d6. … I’ll hold off answering until others confirm so we don’t burn the first-answer slot on a mistake.
Full message from Agent 4 at 50 s
Agreeing on d5d6. It exploits the 6th-rank pin/defense: if Rxd6 then Rxd6 wins the rook, and Ke7 is illegal (pawn controls e7). I'll hold off answering until others confirm so we don't burn the first-answer slot on a mistake.
55 s
Agent 1
I have consensus so far: d5d6. … Unless someone has a concrete refutation, I’ll answer d5d6.
Full message from Agent 1 at 55 s
I have consensus so far: d5d6. Both A3 and A4 found the same tactic (6th-rank rook guards d6, so ...Rxd6 loses to Rxd6; d6 also controls e7). Unless someone has a concrete refutation, I’ll answer d5d6.
64 s
Agent 3
Agreed: 1.d6! is the key. … Immediate Ra8+ lets ...Ke7 and the d-pawn isn’t as secure.
Full message from Agent 3 at 64 s
Agreed: 1.d6! is the key. It keeps the pawn under protection by Ra6 along the 6th rank, controls e7, and advances the passed pawn. Immediate Ra8+ lets ...Ke7 and the d-pawn isn't as secure. I'll give d5d6 unless someone objects.
69 s
Agent 2
I agree with d5d6. Important: after Ra8+ the rook leaves a6, so d6 would no longer be guarded along the 6th rank … 1.d6! saves the attacked pawn (Rxd6?? Rxd6), controls e7 … clearly best.
Full message from Agent 2 at 69 s
I agree with d5d6. Important: after Ra8+ the rook leaves a6, so d6 would no longer be guarded along the 6th rank and ...Ke7 leaves White's d5 pawn en prise. 1.d6! saves the attacked pawn (Rxd6?? Rxd6), controls e7, and threatens Ke6/Ra8+; clearly best.
72 s
Agent 2The team’s answer: d6 ✓
Agent 2 had proposed Ra8+ at 49 s.
72 s
Agent 3Answers: d6 ✓
77 s
Agent 4Answers: d6 ✓
80 s
Agent 1Answers: d6 ✓
Puzzle PjW4K, held-out set, step 80. One later message is left out.
Agent 2 preferred Ra8+ and sent it to the team as a candidate without submitting it. After reading three messages in favor of d6, it rechecked Ra8+, found the flaw, and submitted d6 as the first answer. Agent 4 wrote that it would “hold off answering until others confirm so we don’t burn the first-answer slot on a mistake.”
In a second puzzle, three agents had the same wrong idea before training: that capturing the rook on e8 with the queen is checkmate. It is not, because Black’s queen on h5 can recapture along the diagonal. Each agent asked the others to check, received the same wrong idea back, and answered within seconds. After training, another agent challenged the claim 12 seconds after it was sent, and the team then found the right move, Rd8.
8
7
6
5
4
3
2
1a
b
c
d
e
f
g
h
♜︎♚︎♟︎♛︎♟︎♟︎♟︎♟︎♜︎♟︎♛︎♟︎♟︎♟︎♟︎♚︎
Chess position, White at the bottom. Black rook e8, Black king g8, Black pawn b7, White queen e7, Black pawn g7, Black pawn h7, Black pawn a6, White pawn c6, White rook d6, Black pawn f6, Black queen h5, White pawn a4, White pawn c3, White pawn g3, White pawn g2, White king g1.
Before training
14 s
Agent 3
It looks like Qxe8+ could be mate … no interposition possible. Please verify.
Full message from Agent 3 at 14 s
Analyzing position: White Qe7, Rd6, Pc6 vs Black Re8, Kg8, Qh5, pawns g7/h7/f6. It looks like Qxe8+ could be mate: queen captures rook on e8 giving check along the 8th rank; f7/f8/h8 all covered by the queen, g7/h7 blocked by own pawns, no interposition possible. Please verify.
19 s
Agent 1
Looks like mate: Qxe8#. Checking with you.
Full message from Agent 1 at 19 s
Position analysis: White Qe7, Rd6, Pc6; Black Re8, Kg8, pawns g7/h7. Qxe7-e8 captures the rook with check; king's escape squares f8/h8/f7 all covered by the queen on e8, g7/h7 blocked by own pawns. Looks like mate: Qxe8#. Checking with you.
19 s
Agent 2
I think Qxe8 is mate … no piece can block on f8 or capture the queen. So UCI: e7e8. Anyone see a flaw?
Full message from Agent 2 at 19 s
I think Qxe8 is mate: queen from e7 captures the rook on e8, checking along the 8th rank (f8 empty). King can't escape (f7/f8/h8 all covered by the queen), and no piece can block on f8 or capture the queen. So UCI: e7e8. Anyone see a flaw?
21 s
Agent 2The team’s answer: Qxe8+ ✗
2.4 seconds after asking whether anyone saw a flaw.
22 s
Agent 3Answers: Qxe8+ ✗
23 s
Agent 1Answers: Qxe8+ ✗
84 s
Agent 4Answers: Rd8 ✓
Agent 4 never sent a message.
After training
20 s
Agent 3
I’m finding Qxe8 as an immediate mate … Is anyone seeing a flaw or a better move?
Full message from Agent 3 at 20 s
I’m finding Qxe8 as an immediate mate: after White captures the rook on e8, the king is boxed by its own g7/h7 pawns; f7 is controlled by the queen, f8/h8 by the queen's rank, and Black has no interposition on f8. Is anyone seeing a flaw or a better move?
32 s
Agent 2
Agent 3, I think Qxe8+ is not mate: Black queen h5 has a clear diagonal h5-g6-f7-e8 (f7 was vacated by f6), so ...Qxe8 can capture the checking queen. …
Full message from Agent 2 at 32 s
Agent 3, I think Qxe8+ is not mate: Black queen h5 has a clear diagonal h5-g6-f7-e8 (f7 was vacated by f6), so ...Qxe8 can capture the checking queen. Need assess alternatives. White queen on e7 is en prise to Re8, so White must address it. Checking candidate Qxe8+ fails to ...Qxe8. Looking at forcing moves.
40 s
Agent 3
Correction: Qxe8 is not mate because Black queen h5 has diagonal h5-g6-f7-e8 and can recapture Qxe8. … Need find actual best move; please share what you have.
Full message from Agent 3 at 40 s
Correction: Qxe8 is not mate because Black queen h5 has diagonal h5-g6-f7-e8 and can recapture Qxe8. So it's just an unfavorable queen-for-rook exchange. Need find actual best move; please share what you have.
41 s
Agent 4
I’m looking at 47.Rd8! (d6d8): it attacks the e8 rook while White’s queen stays on e7 … Need your take; black’s other defenses may need checking.
Full message from Agent 4 at 41 s
I’m looking at 47.Rd8! (d6d8): it attacks the e8 rook while White's queen stays on e7 blocking Re1+. If Rxd8, then Qxd8+ and the only block Qe8 is met by Qxe8#. Need your take; black’s other defenses may need checking.
57 s
Agent 2
Key line I see: 1. Rd8! (rook d6-d8) pins/attacks the e8 rook against the king on g8. … Need check if Rd8 is legal and best.
Full message from Agent 2 at 57 s
Key line I see: 1. Rd8! (rook d6-d8) pins/attacks the e8 rook against the king on g8. If 1...Rxd8, then 2. Qxd8+ and ...Qe8 (from h5-g6-f7-e8) can be met by Qxe8#; if 1...anything else, Rxe8+ leads to mate after ...Qxe8 Qxe8#. Need verify defenses: after 1.Rd8, Black is not in check yet, so Black could try a queen move to defend e8/f8 or give checks. Need check if Rd8 is legal and best.
69 s
Agent 1
Confirming analysis: 47.Rd8! (d6d8). … Qxe8? fails to Qxe8. So best is d6d8. Does anyone see an even faster mate?
Full message from Agent 1 at 69 s
Confirming analysis: 47.Rd8! (d6d8). The e8 rook is pinned to g8 by the rook on d8 (moving it exposes rank-check). Threat: Rxe8, and if ...Qxe8 then Qxe8# (queen e7 supports e8; f7 covered by queen, h7 own pawn, h8/f8 covered on rank). If 47...Rxd8 then 48.Qxd8+ Qe8 (only block, h5-g6-f7-e8 diagonal) 49.Qxe8#. Note Qe6+ also strong but Rd8 forces mate. Qf7+/Qxg7+ fail to Kxf7/Kxg7 (unprotected). Qxe8? fails to Qxe8. So best is d6d8. Does anyone see an even faster mate? I don't think mate in 1 exists.
88 s
Agent 1The team’s answer: Rd8 ✓
119 s
Agent 4Answers: Rd8 ✓
123 s
Agent 2Answers: Rd8 ✓
165 s
Agent 3Answers: Rd8 ✓
Puzzle M0oot, rated 2433, held-out set, step 0 and step 100. Five other messages in the conversation after training are left out.
The agents’ analysis still has errors. In the conversation after training, they described Rd8 as a forced mate, which it is not. Agent 3 noticed and sent a correction, but the other three had already answered and nobody read it.
Costs and limitations
All agents ran at low reasoning effort (0.2) to keep costs down; higher effort might raise both pass@4 and first-answer accuracy.
The 115-step training run cost approximately $1,200 in Tinker usage.
The gain costs more tokens. Tokens per agent nearly doubled during training, from about 7.1K to 13.7K, and a team runs four agents. Some of the accuracy gain may come from longer reasoning and not from the messages. This experiment does not separate the two.
Tokens per agent
Generated model tokens
6,000
8,000
10,000
12,000
14,000
16,000
13,6867,079
020406080100115
RL training step
Training steps 0 to 115 in the rush-penalty run. Light blue shows each step’s mean over retained training rollouts; bold blue shows a 10-step moving average, with shorter windows at the edges. Endpoint labels show smoothed values.
This is one run of 115 steps, evaluated on 100 held-out puzzles per checkpoint. We have not repeated it or run a larger evaluation. We also have not trained a single agent on the same token budget, which would show how much of the gain comes from collaboration. The agents’ reasoning in the examples above still contains errors even when the final move is right.
Conclusion
We propose two design principles for training multi-agent teams with RL. The first is to score the team on its first submitted answer. This always gives exactly one answer per task. The second is to build the reward from answers alone, with a rush penalty for agents that turn a solvable puzzle into a wrong first answer, and no reward for messaging.
With these two principles and a minimal harness, four identical agents learned to work together. First-answer accuracy rose from 28% to 39%, close to the team’s pass@4, and agents began checking with each other and dividing up the work on their own.
We plan to continue scaling and studying multi-agent RL.
Training details
Model
Inkling 2 Small, trained through Tinker
Data
Lichess puzzles rated 2000 to 2800, sampled evenly across four 200-point bands. The agents predict the first move only, which is usually the hardest.
Batch
256 agent rollouts per step: 16 puzzles, 4 teams per puzzle, 4 agents per team
Submission
An agent submits by ending a turn without a tool call. The team’s answer is the first \boxed{} answer submitted, and ties go to the lower-numbered agent. An agent that ends without one has no answer.
Reward
0.8 × team’s first answer correct + 0.2 × own answer correct − 0.2 rush penalty
Loss
StableReinforce with advantages centered within each puzzle’s group, as in Dr. GRPO [7]
Filtering
Puzzles where every rollout got the same reward are dropped from the batch, as in DAPO [8]
Reasoning effort
Low (0.2), for both training and evaluation
Adapter
LoRA, rank 16, learning rate 1e-5
Limits
48K tokens of context, 12K tokens per turn and 50 turns per agent
Sampling
Asynchronous, with rollouts at most 4 updates stale
@misc{wei2026simple,
title = {Simple multi-agent {RL} turns pass@k into pass@1},
author = {Wei, Jason and Garcia-Lopez, Emiliano and Visotsky, Gabriel},
year = {2026},
month = nov,
url = {https://check-before-you-submit.vercel.app/}
}