Liquid AI explains on-device small model design: antidoom training cuts doom loops from ~10% to 1% in LFM2.5

helloiamleonie · x · 2026-09-25

A detailed thread by a Liquid AI researcher on how the LFM2 family of on-device models (LFM2.5-350M to 2.6B) is built.

Three constraints: memory (4GB usable on an iPhone 8GB device, must fit weights + KV cache), speed (agentic workflows involve many tool calls and reasoning steps), and quality.
Architecture: efficiency must be designed from the ground up. Small models mix full-attention layers with efficient layers; since theoretical layer gains differ from real-world ones, they profile on target hardware and landed on grouped query attention combined with gated short conv layers.
Embedding problem: large-vocab tokenizers bloat small models' embedding layers, shrinking effective parameter count, so they minimize the embedding layer.
Post-training: modern recipe = SFT → RL plus on-policy distillation and agentic RL.
Antidoom training: to fix doom loops (10%) in small reasoning models, they detect the loop, find the triggering token, reject/downsample it, and upsample alternatives — reducing doom loops to 1% in LFM2.5-2.6B. Also a nice complement to the sparse-reward problem in RLVR.

More from Models