Lafa paper scales flow models to 8B by adapting AR weights, 2.54x AR throughput
burny_tech · x · 2026-10-26
Lafa: continuous flow language models reach 8B parameters
Researchers from UC Berkeley, Amazon, KAIST and CMU introduce Lafa, showing continuous flow models can scale to 8B parameters without training massive diffusion LLMs from scratch.
Method
- Blockwise flow model: causal attention on the prefix, bidirectional asynchronous flow within each block
- Adapts pretrained autoregressive (AR) models (0.5B–8B) with under 30B tokens, more data-efficient than prior diffusion LMs
- Flow training preserves or improves AR capabilities via a multi-token prediction effect, leveraged in a self-speculative decoding scheme
Results
- Matches or surpasses AR fine-tuning on instruction following, math, reasoning and coding benchmarks
- 2.54x the throughput of an AR model at comparable quality, beating multi-token prediction and discrete diffusion
- Fast inference via prefix caching and blockwise parallel decoding; code and models are open-sourced
The findings position continuous flows as a scalable foundation for language modeling and a post-training path to high-throughput inference for production agents.
Related event: Lafa Scales Continuous Flow Language Models to 8B with 2.54x Throughput(2 posts)→
More from Research
- CoRL paper: one-step inference matches multi-step for large VLAs, 1.6-4.8x faster action generation — kaixhin · 2026-10-26
- Why biology is hard to simulate: universally sloppy parameter sensitivities, per classic PLOS paper — rbhar90 · 2026-10-26
- LessWrong post argues FDT is difficult-to-impossible to specify due to logical counterfactuals — jessi_cata · 2026-10-26
- Bitter Lesson misread: scaling won't brute-force biology, researcher argues — anshulkundaje · 2026-10-26
- Chain Theory: rebuilding physics from quantum information, AI finds 137 fine-structure-constant expressions — skdh · 2026-10-26
- HKU/ZJU/Fudan with Lightspeed launch HiSTEVE: hierarchical VLA agent playing Minecraft via native mouse and keyboard — tencent · 2026-10-26