Bolmo recipe: short additional training retrofits models to bytes
allen_ai · x · 2026-10-07
AI2 described its recipe: a short additional training run adapts existing subword models to bytes while keeping the model's core.
New components group bytes into variable-length patches for the model to process, then expand outputs back to byte-level representations to predict the next byte.
Related event: AI2's byte-level language model Bolmo published in Nature with open weights(6 posts)→
More from Research
- From Feelings to Metrics: COLM 2026 paper turns LLM vibe-testing into structured evaluation — boknilev · 2026-10-08
- Berkeley's RPG lets robots watch demos and practice to improve, hitting 30/30 on real tasks — berkeley_ai · 2026-10-08
- Open-source AI-SQL engine Quail adds prefix sharing: 3x fewer tokens, 2.6x faster queries — sh_reya · 2026-10-08
- Unified DiT initialized from Qwen3-1.7B trains fast with 64x spatial compression — ostrisai · 2026-10-08
- OpenAI's 722 math results show striking lack of crypto breakthroughs, fueling censorship theory — burny_tech · 2026-10-08
- Google DeepMind partners with Chan Zuckerberg Initiative on multimodal biology datasets — pushmeet · 2026-10-08