Photonic Training of a 1B GPT-2 Without Backprop Lands in PNAS
IgorCarron · x · 2026-10-14
Igor Carron shares his team's new PNAS paper: an experimental implementation of direct feedback alignment (DFA) on a hybrid electronic-photonic system, pre-training a 1B-parameter GPT-2 with random projections and bypassing backpropagation entirely.
Key points:
- Backprop remains the major compute and energy bottleneck of modern deep learning training
- Photonic approaches offer low power and high speed but have so far been limited to inference and basic tasks
- This work demonstrates a versatile, scalable optical training path pushed to the 1B-scale language model regime
Carron also reflects that research, like startups, requires an element of faith — quoting advice to avoid "smart skeptics" who excel at objections but can't execute and demoralize believers.
More from Infra
- SK Hynix has struggled with HBM4 back-end issues for over a year, Samsung tipped to lead HBM market next year — zephyr_z9 · 2026-10-14
- Startup Builds $18,000 Local AI Server With 128GB VRAM, Runs 16 Concurrent Sessions — bendee983 · 2026-10-14
- A used dual-RTX-3090 local LLM rig that isn't much faster — why? — jonejy · 2026-10-14
- Xinjiang's 167 GW green power, 16ms to Chongqing: the grid edge in AI race — 96Stats · 2026-10-14
- Migrating a key API from Node to Bun cut p99 latency 80% and memory 63% — DanielLockyer · 2026-10-14
- Agent traffic is a strange inference workload: long prefix caching beats raw tokens per second — rohanpaul_ai · 2026-10-14