AI practitioners clash over fast weights: is runtime LoRA adaptation a dead end?
HankYeomans · x · 2026-10-11
A debate over a fast-weights attempt using LoRA. @danveloper argues nearly everyone in AI tried this in 2023 and abandoned it: personalized facts are never dense enough to dent the weights, which is why RAG won and markdown/wikilinks plus graphs are the current flavor. Fact retrieval works best with in-context learning, and runtime-swapped adapters trained in the background are a dead end; LoRA may help only with evolving personalized preferences. HankYeomans pushes back that "it's been tried before" is no reason to stop building.
More from Models
- Anonymous unreleased AI model builds impressive pure-code three.js in hours — karminski3 · 2026-10-11
- Researcher Breaks Qwen 2.5 via Endless Gaslighting, Forced Off arXiv by Endorsement Rule — IndraVahan · 2026-10-11
- Outside CVP/Daybreak, the world's best cybersecurity model is Chinese GLM 5.3, not Claude — zephyr_z9 · 2026-10-11
- Was Claude's gibberish fixed by capping KL divergence in RL? One theory — burny_tech · 2026-10-11
- 20-year engineer benchmarks Gemma4-31B vs Qwen3.8-27B locally; GPT-6.1-Sol is still another tier — therealjerseytom · 2026-10-11
- Rumor: Google employees claim internal model Carbon beats unreleased Gemini — bindureddy · 2026-10-11