Stanford CS312 experiment: looping transformers lose; mid-training untying recovers 61% of gap

tatsu_hashimoto · x · 2026-11-19

Tatsu Hashimoto published final validation losses from a CS312 quiz experiment: Dense 2.9333 beats Untied-midway 2.9805 and Looped 3.0538 — untying midway recovers 61% of the gap.

Key takeaways:

The attached image shows the full quiz answer key with loss comparisons and the weight-sharing sweep.

Original post →

More from Research

Research channel →