Stanford CS312 Quiz: What to Tune When Batch Size Grows 8x

Stanford CS312 instructor Tatsu Hashimoto posted a predictive quiz on optimization intuition: with batch size scaled up 8x under a fixed token budget, the answer is that weight decay should be scaled by 8 as well.

2026-10-28 ~ 2026-10-28 · 2 related posts

• Stanford CS312 quiz: can LR or weight decay rescaling compensate for an 8× larger batch? — tatsu_hashimoto · 2026-10-28
• Stanford CS312 quiz: scale weight decay 8x when batch size grows 8x — tatsu_hashimoto · 2026-10-28