LoRA From Scratch: recovering a hidden weight change with two thin matrices
dima806_dima · x · 2026-11-03
Dima Yakubovskyi's 7-minute tutorial "LoRA From Scratch" demonstrates the mechanism hands-on: hide a change of known size inside a small network, then watch a pair of thin low-rank matrices recover all of it while the rest of the model stays frozen — a concrete way to grasp why two small matrices can replace million-weight updates.
More from Research
- Qwen's Astra gains come from an agentic ML loop, not weight introspection — petrusenko_max · 2026-11-03
- Nature: Connectome simulations identify a three-neuron circuit generating fly walking rhythms — patrickmineault · 2026-11-03
- AI researcher: models editing weights is real but the self-improvement hype overclaims — paul_cal · 2026-11-03
- D&A Turns Any VLM into a Zero-Shot Robot Reward Model, SOTA on 3 Benchmarks — DJiafei · 2026-11-03
- CT-Perturb: LLMs Reach 82% of Full-Data Cell Perturbation Prediction with 10% of Measurements — david_van_dijk · 2026-11-03
- Christian Kroer's Book 'Games, Markets, and Online Learning' Now in Print, Free PDF Available — bremen79 · 2026-11-03