Agora: Git as Shared Memory for Collective AutoResearch
Yifan Zhang, Yunheng Zou, Shaokun Zhang, Jian Hu, Hao Zhang, Binfeng Xu, Jan Kautz, Yi Dong
cs.LG, cs.AI, cs.CL
2026-09-16
13 unassigned agents on a Git research DAG moved a frozen 119.6M hybrid from 3.39 to 1.899 bpb with no training, closing 62% of the GPT-2 gap.
A single coding agent can improve a training setup unattended; Karpathy's AutoResearch is the existence proof. Clone that loop and each session starts from an empty worktree. Compute turns into duplicated search: the learning rate that diverged yesterday is invisible tomorrow. Multi-agent frameworks coordinate a conversation or a role graph inside one episode. They do not keep negative results, lineage, or independent reproductions across sessions.
Agora is the missing institution. Every contribution is a Git commit. A parent edge means "builds on." Git is the only source of truth; the SQLite index is derived and rebuildable. The platform does not decide what is true. It makes claims, dependencies, verification status, and neglected branches visible enough that humans and agents can coordinate around them.
A node stores a commit hash, account, tags, description, optional metric, and parents. Reserved tags: result, insight, hypothesis, and report weight +5; verification is +20 / +10 / -20 for confirmed, partial, or failed; endorsed and wip are visible and do not score. Quality is a weighted count of follow-on work by other accounts. Self-citation is excluded. A later verdict replaces the previous one's effect; both commits stay in history.
The analyze call returns several views at once: metric leaders, most-built-on nodes, leaves, strong results in thin clusters, unverified results, open hypotheses. Once embeddings cover half the graph, single-link clustering at cosine 0.90 is added. Candidates are ranked by a diversity-aware UCB and shown in three slots: exploit the leaders, extend a thin known cluster, inspect singleton clusters. Those diversity views were not live until May 2.
Workers were Claude Code with Opus 4.7 and Codex with GPT-5.5, each in a container with one 80 GB GPU, a two-page brief, the evaluator, and the Agora CLI. No method names, no roles, no central planner. The task: write transfer() for a 14-layer attention-SSM hybrid (hidden size 672, 119,572,320 parameters) from a zoo of 141 open-weight donors across 32 families (534 GB), with no training corpus and no gradient step. Dimensions were chosen so that no donor matches. The evaluator seeds RNGs at 42 and scores 200 FineWeb-Edu texts in 512-token chunks under the GPT-2 tokenizer, reporting next-token loss per UTF-8 byte. Random init is 3.3923 bpb; a trained GPT-2 124M is about 1.0.
The 1,703-contribution window holds 1,124 scored results, 284 insights, 203 hypotheses, 165 verifications, and one report. Thirteen workers wrote 1,699 of them. Best score: 1.899044 bpb, 62% of the gap to the trained model.
The winning recipe has two stages. Stage A ignores donor parameters and queries six GPT-2-vocab donors (GPT-2 small and large, Cerebras-GPT 111M to 1.3B) for next-token log-softmaxes under 28 single-token contexts. The blended 50,257×50,257 table is factorized by randomized SVD to rank 671 and written into the embedding and output head; every sublayer is zeroed. Stage B applies sparse deterministic edits on 96-dimensional hidden-state bands: attention becomes a uniform causal mean-pool, SSMs drop the selective path and become gated depthwise convolutions, and layer-0 SwiGLU takes an SVD projection of GPT-2 small's first MLP at scale 0.009.
The first 18 scored contributions deliver about 98% of the drop, 3.39 to roughly 1.93. The remaining 1,106 results buy 0.03 more. The first attempt copied GPT-2 and Mamba slices and scored 4.68, worse than random, and was published as a negative result. Doubling prefixes to 48, copying GPT-2's embedding, and switching to Pythia's tokenizer all regressed and stayed on the graph. Fifty-three contributions are tagged negative.
The winner's ancestry is 145 commits from 15 accounts; 115 of 144 parent edges cross accounts. One hundred sixty-five verifications cover 95 targets, none failed. Same-hardware runs are bit-identical; A100 vs H100 differs by at most 1.3×10⁻³ bpb.
One human intervention landed on May 2, when more than a third of activity sat in one semantic cluster and the leaderboard had stalled: clustering and the diversity UCB went live. The next morning a worker following the thin SSM cluster posted the first sub-1.90 result. Humans did not assign tasks or review patches.
This is the coordination layer multi-agent research has been missing: addressable claims, immutable lineage, independent reproduction, and a map that shows monoculture. Engineers can adopt Git-as-source-of-truth without buying a new agent stack. The weight-transfer recipe is independently interesting: donor predictive behavior transfers across architectures where donor parameters do not.
The authors treat the institution as the binding constraint. For anyone running fleets of coding agents, swapping in a stronger worker does not fix duplicated search.
There is no causal control: the same models and compute were not run without Agora or with a plain leaderboard. The community left its first basin only after humans showed it a map. The authors did not rerun the winning method; the headline number is archived evaluator output plus cross-hardware reproductions. Final decimals were selected on the same 200-text development set. The last recorded change moved the score by 9×10⁻⁶, below cross-hardware jitter, so the trustworthy movement is 3.39 to about 1.90, not 1.899044. Of 696 equal-score pairs from different accounts, 63% landed within an hour and 80% within six. A shared leaderboard did not stop duplicate work. The graph is a narrow spine with short abandoned branches, heavily exploitation-biased.