Stanford/EPFL Paper: Coding Agents Train Optical Neural Networks In Situ, Near-Backprop Accuracy
GordonWetzstein · x · 2026-11-20
"In Situ Training of Optical Neural Networks by Coding Agents" (arXiv:2611.23875, Stanford & EPFL, incl. Gordon Wetzstein).
- Problem: Optical neural networks are fast and energy-efficient, but every device departs from its design and must be trained with the hardware — traditionally via a calibrated per-device digital model or gradient-free trial and error.
- Method ("optics as code"): A coding agent trains the optical network from measurements alone; a harness returns camera images of its probes and requires a hypothesis per request. The same harness also trains an optical image encoder.
- Findings: The agent forms a physical understanding of the optics and reparametrizes the network with interpretable optical elements tuned on hardware. It beats gradient-free methods on classification and image encoding, approaching backprop through a calibrated model at a fraction of the cost.
- Implication: No per-device calibration or hardware gradients needed, and results improve with more capable models — a practical route to scalable optical neural networks and a general tool for calibrating optical experiments.
More from coding & agent
- Your old CLAUDE.md is hurting new Claude models; author trimmed it to 36 rules-free lines — PrajwalTomar_ · 2026-11-20
- Open-sourced Claude AI team: PM, CTO, UX roles that challenge your product ideas — marilynika · 2026-11-20
- Open-source Intern harness: Opus 5.5 hands off to Hy4 Flash, 92.4% at 1/3 the cost — m4rkGS · 2026-11-20
- Dev shares MCP lessons: human-only publishing enforced by omitting the tool entirely — mariobalca · 2026-11-20
- Optics as Code: Stanford agent tunes optical neural nets from 35% to 96.8% via hardware-in-loop experiments — GordonWetzstein · 2026-11-20
- Gordon Wetzstein: coding agent in the loop boosts optical NN accuracy 35% to 96.8% — GordonWetzstein · 2026-11-20