18-year-old independent researcher shows self-repairing training converges, validated in 383 runs
Cohere · youtube · 2026-11-14
Cohere Labs hosted a talk by Rokib Al Dhin Raadh, an 18-year-old independent researcher, whose input-to-state stability analysis of self-repairing training was accepted at DeepMath 2026 after double-blind review.
Key findings
- Standard convergence proofs assume every update is a gradient; the ORMAS architecture's self-repairing updates violate that assumption yet still converge.
- Zeroing a converged conv layer at epoch 100: a parameter-matched baseline stays at chance (10.0%±0.0%), while the self-repairing network recovers to 80.3%±1.6% via 85 bounded, logged corrections.
Theory
- Training is modeled as a switched dynamical system (contractive gradient flow, bounded disturbance, isolated resets), yielding the ISS bound limsup‖θₜ−θ‖ ≤ γ = 2βM/α + εmax/(ηα) under local L-smoothness and α-strong convexity.
- The bound is non-vacuous only if correction magnitude decays; across 383 evaluation runs, corrections shrank by 83–93%, making it a falsifiable prediction.
Theory-practice gap
- The per-step contraction holds on only 31.6% of steps yet the run converges; the offset term never vanishes, so the result is practical rather than strict ISS, and forward invariance remains unproven.
- Open problems: deriving ε(t) from first principles, relaxing strong convexity to Polyak–Łojasiewicz, dwell-time conditions for discrete reinitializations.
Preprint available on Zenodo.
More from Research
- SFT-ing small models for weight-space programming hits 99.8% on logit lens — lateinteraction · 2026-11-14
- Third-person self-talk regulates emotion without cognitive control, fMRI study shows — AnnaCiaunica · 2026-11-14
- PiHand: latent hand-object interaction priors cut dexterous-hand RL training data 10x — KyleMorgenstein · 2026-11-14
- New paper: symmetry as a common origin of hierarchical emergence and abstraction — mjdramstead · 2026-11-14
- MoE combine traffic cut from 16 to 8.25 bits per value via RHT + S2F6 quantization — snowclipsed · 2026-11-14
- Scientists simulate an entire living cell in 4D, published in Cell — tomchapin · 2026-11-14