AI2's byteification method lands in Nature, extends Bolmo to Qwen and Llama
allen_ai · x · 2026-10-07
The Allen Institute for AI announced that its Nature-accepted paper on retrofitting LLMs to operate directly over bytes — the approach behind Bolmo — with a two-stage "byteification" procedure that converts subword-based models into byte-level ones approaching their performance.
The motivation: subword tokenization obscures fine-grained character-level information, hurting code, math and biological sequences. AI2 also released new checkpoints extending the method from Olmo to Qwen and Llama.
Related event: AI2's byte-level language model Bolmo published in Nature with open weights(6 posts)→
More from Research
- Meta, DeepMind and Isomorphic join DOE/NIH Virtual Biology Initiative to model the cell — paulnovosad · 2026-10-08
- Is formalized code as verifiable math legit, or does it sound like a scam? — ziv_ravid · 2026-10-08
- Public Sector's Highest-Value Move: Build Big Datasets, Let AI Infer the Rest — Afinetheorem · 2026-10-08
- 9B and 0.6B Embedding Models Share One Vector Space for Encode-Small-Search — tomaarsen · 2026-10-08
- OpenAI Researcher: Physical Science Progress Will Outpace Expectations on Compute and Simulation Overhang — sschoenholz · 2026-10-08
- Dataset for one study would cost $400,000 under 1/sqrt(k) data scaling — generativist · 2026-10-08