Code-switched questions hurt LLM accuracy; CANVAS steering fixes internal representations
burkov · x · 2026-10-31
A new study investigates why multilingual LLMs answer mixed-language (code-switched) questions less accurately than monolingual ones, and whether the gap traces to how models internally represent grammatical structure.
The researchers built controlled comparisons: 955 factual queries across 9 target languages and multiple model families, pairing identical information expressed with either the source language's or the target language's grammatical frame. They then compared internal representations and answer quality across conditions.
They also introduce CANVAS, a method that uses information from the mixed-language input itself to steer the model's internal representations toward the target language, improving answer accuracy.
More from Models
- Dev asks for a mathematician: Claude's Muon optimizer theory is too deep to verify — vxnuaj · 2026-10-31
- Claude's Muon optimizer theorem checks out numerically to 4 decimal places — vxnuaj · 2026-10-31
- KOL jokes Codex is great — if you're a rich American — lxfater · 2026-10-31
- Early tester calls Gemini 4 Argon the most free-range model since gpt-4-base — amplifiedamp · 2026-10-31
- User maxes out both Claude sub and $200 Codex sub days before cycle ends — CtrlAltDwayne · 2026-10-31
- Rumor: OpenAI delayed new model over alignment issues, more than any other lab — haider1 · 2026-10-31