7 PhD students train a 7B model from scratch in one summer using hundreds of agents

量子位 · wechat · 2026-09-14

Seven PhD students at Beijing Zhongguancun Academy trained a 7B model, ZGCM-1, from scratch in one summer, open-sourcing data recipes, code, checkpoints, and logs. It leads same-size models across 14 reasoning benchmarks and rivals Qwen3-235B-A22B on some math/search tasks.

Hundreds of agents as teammates: Agent sub-teams handled data cleaning, experiment monitoring, fault diagnosis, and evals, with a ZGent platform persisting decisions and debug experience as reusable Skills. Humans set goals and made key calls.
Autonomy audit: On an L1-L5 scale, experiment monitoring and deployment hit L4; architecture and algorithm design stayed at L2.
Hard-won findings: A mysterious capability regression despite falling loss was traced to data sharding/shuffle drift; they built an ACE probe suite (18 categories, 183 abilities, 2503 probes). Stricter SFT filtering cut samples by 44.9% yet improved results; too much long-CoT data hurt instruction following.
Efficiency: Hybrid local+global attention scaled context 16K→256K with 3.94x throughput and 1/6 KVCache vs full attention; Muon+FP8 training hit equal loss 4.2x faster than BF16/AdamW.

The team is now scaling to 400B-500B models.

Original post →

7 PhD students train a 7B model from scratch in one summer using hundreds of agents

More from coding & agent