Trick: Adam with betas=0 and lr=ε=10,000 acts as an exact gradient observer
anpaure · x · 2026-10-29
Origin of anpaure's optimizer forensics thread: run Adam and Dimuon from identical weights on the same batch, set Adam's betas to 0 and lr = ε = 10,000 — its update becomes ≈ −G, and inverting Adam's formula recovers G exactly. This enables a general measurement scheme: use an "Adam gradient observer" to recover G from ΔW, predict the target optimizer's update, and repeat with fresh observers to quantify disagreement — revealing the hidden transformation any optimizer applies to gradients.
More from Research
- New paper challenges JEPA feature-selection theory: unpredictable features can be kept — andreea7b · 2026-10-29
- SREGym to debut at Bench Summit, targeting AI SRE evaluation frontier — tianyin_xu · 2026-10-29
- AA-Robotics benchmark: GPT-6 Astra tops robot-arm control, still ~100x slower than humans — ArtificialAnlys · 2026-10-29
- Delta-Matching closes the FP8 attention gap: all 7 core matmuls in FP8, loss matches BF16 — songhan_mit · 2026-10-29
- RFD4-Proteina lands on Hugging Face as HF publishes its October AI for Science roundup — AllThingsApx · 2026-10-29
- Google's PLE: 85 real engineering tasks, best agent config scores only 50.6% — rohanpaul_ai · 2026-10-29