will cb: the circuit view dissolves most of deep learning's supposed mystery
willcb · x · 2026-10-16
Extending his circuit-view thread, will cb argues most of deep learning's supposed mystery disappears once you embrace circuits. Circuits are expressible in bits; bit-truncated expressions approximate full ones well on most inputs with diminishing returns. Going fp4→fp32 barely hurts expressivity — it's more about optimization dynamics — and RL finds the high-order bits first.
Related event: Researcher Argues Circuit View Dissolves Deep Learning Mysteries(4 posts)→
More from Research
- Ex-OpenAI researcher claims most neurons are legible now; critics call it hype — anshulkundaje · 2026-10-16
- Physicist pushes back on ML community's acceptance that models may be beyond understanding — tszzl · 2026-10-16
- Survey: AI agent evaluation is getting realistic but still fails to measure safety and reliability — burkov · 2026-10-16
- COVA-SFT dataset trains models to solve text problems with visual chain-of-thought — yoavartzi · 2026-10-16
- USTC paper uses speculative sampling to cut latency in interactive video generation — ustc · 2026-10-16
- Tsinghua's CoTranslator distills reverse CoT to boost 4B model math scores — Tsinghua · 2026-10-16