Qwen vs. Gemma: Tokenizer Efficiency Explains Coding Performance Gap
WhoRoger · reddit · 2026-08-09
A developer found a massive discrepancy when feeding the same 330 lines of HTML/JS code to Qwen 35B and Gemma 26B: Qwen tokenized the input to 1,609 tokens, while Gemma produced 4,258 tokens.
This significant difference in tokenization efficiency helps explain why Qwen is regarded as better at coding, while Gemma leans towards language tasks. Qwen can process code more efficiently as a specific structure, whereas Gemma breaks it down into smaller pieces like regular language. The author wonders if retraining Gemma with a more efficient tokenizer could help bridge this gap.
More from Models
- willcb: Opus 5.5 doesn't smell bigger, it smells like they fixed the data — willcb · 2026-10-16
- Unverified claim: Apple and Meta's 816B-parameter Muse Spark Open runs on two Mac Studios — TansuYegen · 2026-10-16
- Gemini 4 model card: model eliminated months of trial-and-error on bioweapons research — Puzzleheaded-King584 · 2026-10-16
- Every AI picks 7: Anthropic engineer's rule for escaping model taste convergence — rubenhassid · 2026-10-16
- Brain teaser: strip all attention, MLP and embedding matrices from a Llama 3 — how many trainable params remain? — Sauers_ · 2026-10-16
- Lean v4.35.0 ships verified kernels — one written and proven consistent by Claude — geoffreyirving · 2026-10-16