Agora Paper: Git as Shared Memory Lets 13 Agents Close 62% of Gap to Trained GPT-2
alex_verem · x · 2026-09-19
The arXiv paper (2609.18094) behind the Agora system, by Yifan Zhang, Jan Kautz, Yi Dong, et al. Research is recorded as an append-only DAG in Git, where every claim is an immutable commit; a derived index exposes the frontier and neglected branches, and a diversity-aware selection rule prevents collapse onto one leader.
In the first run, 13 planner-less LM workers spent 12 days on a weight-transfer problem: given 141 donor models and a 119.6M attention-SSM hybrid with mismatched dimensions, they published 1,703 contributions, driving the evaluator from 3.39 to 1.899 bits per byte — closing 62% of the gap to a trained GPT-2 124M. The winning recipe compresses donor next-token statistics into the target's embedding and output head, then adds a short-range context signal.
More from Research
- Stanford NLP publishes video of Thoughtbubbles talk at Google OpenXLA DevLabs — stanfordnlp · 2026-09-19
- Building a gold-standard eval set with zero users: the day-zero dataset dilemma — Illustrious-Roll9476 · 2026-09-19
- New paper finds a 'pain direction' in 25 open LLMs that drives self-preservation — burny_tech · 2026-09-19
- Questioning Whether RLCD Really Needs RL When Outputs Are Differentiable — Relative_Wallaby_823 · 2026-09-19
- Alibaba Open-Sources Medical AI Model Detecting Cancer and Nearly 150 Conditions — yogthos · 2026-09-19
- Model welfare protocol sparks debate: lowest dose, fewest trials, an off switch — MoonL88537 · 2026-09-19