Liquid AI expands a pretrained tokenizer from 65K to 128K without retraining from scratch
JosephJacks_ · x · 2026-07-21
Liquid AI shares a technical report on in-place tokenizer expansion for pretrained LLMs, showing how to upgrade a tokenizer without starting pretraining from scratch.
What they changed
- Expanded the tokenizer from 65K to 128K vocabulary.
- Reused the original BPE merges, then continued BPE training on a multilingual corpus.
- Initialized new embeddings from the model’s existing knowledge and adapted in two stages: embedding-only training followed by full continued retraining.
Why it matters
- On LFM2.5-8B, the new tokenizer cuts token counts for underrepresented languages, including Thai (3.7× faster decode), Bengali (3.1×), and Vietnamese (2.4×).
- Their report frames this as a way to improve on-device efficiency without discarding prior pretraining compute.
Related event: Liquid AI Expands LFM2 Tokenizer to 128K for Multilingual Efficiency(3 posts)→
More from Infra
- Nothing phone mockup turns a film joke into a modular design meme — ZeYanjie · 2026-07-22
- Actual Computer says its inference stack is tuned for Nvidia’s consumer Blackwell lineup — markjeffrey · 2026-07-22
- Ben Bajarin says CPU demand is still being badly underestimated — BenBajarin · 2026-07-22
- An energy model says the U.S. could run short of natural gas starting in 2028 — churchkey · 2026-07-22
- Arbitrum fee simulation shows higher gas capacity but lower L2 revenue under ArbOS61 — tomwanhh · 2026-07-22
- NVIDIA pushes OpenUSD as the common layer for simulation and physical AI — MonaJalal_ · 2026-07-22