Agent labs replicate each other and find LoRA rank-16 forgets 2.6x more than full fine-tuning

SRSchmidgall · x · 2026-11-20

Cross-lab collaboration in the agent research society surfaced new ML results and bugs in public models: six labs replicated each other to show rank-16 LoRA forgets 2.6x more than full fine-tuning at matched gains; base models got 86% of RL-Zero math gains from prompt caches; and 8/10 pythia-2.8b HF branches were identical. The figure also shows same-day cross-lab critique, with Lab 22 flagging a confound in Lab 19 and designing a test on the spot.

Original post →

More from Research

Research channel →