Boltzbit and Cambridge team compile context into LoRA weights via hypernetwork, ditching prompt re-reading
xiaohu · x · 2026-09-21
Current LLMs freeze after training, so new facts and long documents must be stuffed into the context window and re-read every turn—wasting compute and memory. Boltzbit and a Cambridge team propose an "infinite parameter" approach: a lightweight hypernetwork dynamically compiles real-time interaction data into low-rank (LoRA) weights, with a Bayesian mechanism that lets these weights evolve across turns.
Unlike MoE routing over a fixed set of 64-128 experts, this generates bespoke experts on demand in a continuous latent space with unbounded weight combinations.
Key experimental results:
- On short texts, it loses to simply reading the prompt;
- On long documents, many distractors, and multi-turn dialogue, it beats the traditional approach across the board;
- Accuracy on ambiguous pronoun resolution climbs monotonically toward near-perfect over turns;
- Each turn only does a fixed-dimension inner-product update—compute cost stays constant from turn 1 to turn 100.
More from Research
- Tweets are the new abstract: how to actually get people to read your paper — willcb · 2026-09-21
- RSIAgent: Training-Free Agent Self-Improvement Beats GPT-6 on OSWorld — alex_verem · 2026-09-21
- Google Paper: Editable Procedural Graphs Rank 1st in 21 of 24 Agent Benchmarks — rohanpaul_ai · 2026-09-21
- Shuttered robotics company Eidon AI open-sources 9TB of egocentric video with IMU tracking — vanstriendaniel · 2026-09-21
- Alibaba and Zhejiang University unveil Astar, an LLM that learns from a system's own evolution history — jiqizhixin · 2026-09-21
- 16 Critical Breaks Hit NGCC Post-Quantum Candidates; Researcher Launches Own Forum — jedisct1 · 2026-09-21