The best LoRA explainer video yet: rank constraints meet intrinsic dimension
sophiamyang · x · 2026-10-27
juliarturc released a deep-dive video on the geometry of LoRA, showing how the rank constraint shapes the search space and connecting it to a task's intrinsic dimension. Chapters cover whether fine-tuning is still needed, geometric intuition for rank, training along random directions, intrinsic dimension of pretrained LLMs, and a demo fine-tuning a query router with Qwen3. Sophia Yang calls it the best LoRA explanation she's seen.
More from Research
- Open Models Match GPT-6 on MedMCQA but Trail 17 Points on Hard Clinical Cases — keunwoochoi · 2026-10-27
- Health-Focused Benchmark Reranks LLMs: Muse Spark Open Jumps from #9 to #1 — keunwoochoi · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27
- Goodfire commits $10M to a 'Human Genome Project for AI' to reverse-engineer frontier models — CatAstro_Piyush · 2026-10-27
- π0.5 study: raising demos from 10 to 438 lifts Place Ball in Bowl success from 7% to 83% — stingning · 2026-10-27
- HandsOn benchmark: 66 verified tasks and 18,000 human demos for dexterous manipulation — stingning · 2026-10-27