On-device continual learning with Online-SDFT self-distillation
burny_tech · x · 2026-08-20
Researchers introduced Online-SDFT, enabling continual learning of Small Language Models (SLMs) directly on phones. Using self-distillation, the method treats the current model as a teacher, utilizing user interaction hindsight to generate better training targets for the student model. This allows the model to evolve in real-time based on user edits and choices without uploading private data, addressing the challenges of sparse interactions and limited compute on-device.
More from coding & agent
- Decision models match LLMs on routing and safety — and are 7-8x faster — randal_olson · 2026-10-16
- Temporal Serverless Workers on Cloud Run: queue-driven autoscaling, scale to zero — rseroter · 2026-10-16
- Meta's Superintelligence Lab Runs 200+ Direct Reports—A Lesson for Flat Multi-Agent Systems — peterjliu · 2026-10-16
- LangChain CEO: the most interesting agent harness work lives at the harness-infra boundary — hwchase17 · 2026-10-16
- GitSwarm paper: agents share branchable Git memory, solving all 30 IMOProofBench problems in one run — anirudhg9119 · 2026-10-16
- Hands-on with Gemini 4 Argon: faster than Opus 5.5 but hallucinates more and costs more — prasenx · 2026-10-16