Multi Agents

multi-agents

Multi-agent teams for real work: measured, not assumed.

Docs ·
Quickstart ·
Teams ·
Benchmark ·
Roadmap
North star: most agent orchestrators skip one question, and the benchmark here answers it:
did the team beat a single agent on the same task, and what did the extra agents cost?

Run a team of coding agents on a task in your repository: a single agent, a coder ⇄ reviewer
loop, or several agents with a judge. Workers are the coding CLIs you already use (Claude Code,
Codex) or a built-in agent that can use any model through the OpenAI Agents SDK. Each worker
gets its own git worktree. You get one patch back, plus an itemized record of every agent call:
tokens, cost and seconds. Your working tree is untouched unless you pass --apply.

Why measure

The evidence on teams of coding agents points both ways:

- A 260-configuration study found multi-agent setups 2–15% *worse* than single agents on
  SWE-bench Verified (Towards a Science of Scaling Agent Systems).
- A team with a coordinator, a researcher, an engineer and a reviewer reached 72.2% on SWE-bench
  Verified, 7.2 points *better* than mini-SWE-agent but only 0.4 above the strongest single-agent
  baseline, with different models per role (Agyn). A
  decentralized pair, a patch editor and a validator with isolated context, gained 3.6 and 8.4
  points over mini-SWE-agent (icat-agent).
- The best-known coding benchmark is contaminated: OpenAI stopped reporting SWE-bench Verified
  in February 2026.

So the answer depends on the topology, the models and the task, and published numbers may be
contaminated. This project measures it on tasks created after the models' training cutoffs, with
paired comparisons and cost.

Quickstart

Requirements: Python 3.10+, uv, git, and at least one logged-in
coding CLI…