GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
Konstantin Dobler, Federico Scozzafava, Jonathan Janke, Mohamed Ali, Simon Lehnerer
cs.CL, cs.LG
2026-08-14
A controlled study across nine models and eleven languages finds native-language reasoning costs only 1.3-4.1pp on average and crosslingual transfer is strong, but specific model-language combinations collapse capabilities in other languages, down to -19.2pp, demanding broad evaluation after training.
RLVR with GRPO is the dominant recipe for turning pretrained models into reasoners, but the literature is almost entirely English-centric. For non-English and multilingual training the open questions have no systematic answers: should the model reason in English or in its native language, should training be monolingual or multilingual, and can training in one language quietly break capabilities in another. Prior results conflict. Some papers report severe drops from forcing native-language reasoning; others argue native reasoning helps low-resource languages. This study runs the full matrix under one controlled setup.
Nine base models: Qwen3-Base (1.7B/4B/8B), Qwen3 (1.7B/4B/8B), gemma-3 (1b-it/4b-it), and SmolLM3-3B. Eleven training languages (English, Chinese, German, French, Spanish, Russian, Japanese, Thai, Swahili, Telugu, Bengali) plus a multilingual mix, with three held-out languages (Italian, Korean, Portuguese). Up to 23 trained variants per base model.
Training is DAPO-style GRPO without a KL penalty, 500 steps, 64 prompts by 8 rollouts per step, learning rate 1e-6. The reward is binary correctness plus two auxiliary rewards at half weight each: a format reward and a reasoning-language reward, set either to "reason in English" or to "reason in the prompt language" (native). Training data comes from the Multilingual Reasoning Gym, procedurally generated math and logic puzzles spanning 62 training tasks and 30 held-out tasks. Evaluation is avg@8 across MGSM and PolyMath, grouped into Easy Math and Hard Math.
Native versus English reasoning first. English reasoning rewards win on average by 1.3 to 4.1 percentage points across models, but the gap is far from uniform: Bengali, Telugu, and Swahili show large English-reward advantages, while French, Chinese, Japanese, and German are nearly tied. The double-digit native-reasoning drops reported in earlier work do not appear here.
Crosslingual transfer is the second block. Most of the same-language gain transfers to other languages. On French tasks, Spanish-only training recovers the target-language gain of 25.6pp to within 1.0pp. Low-resource languages behave differently: Swahili training improves Swahili evaluation by 32.2pp on average, while the next-best transfer language, Telugu, delivers 20.5pp. Target-language data is irreplaceable at the low end. English is often not the best transfer source; for Qwen3-4B-Base and Qwen3-8B, Bengali training transfers best to Chinese.
The third block, regressions, is the most valuable finding. On Hard Math the Qwen3 instruct family regresses broadly: 1.7B drops 12.0pp in the training language and 14.3pp crosslingually; 4B drops 15.7 and 20.7; 8B drops 14.1 and 18.1. The Qwen3-Base family gains 50-61pp on Easy Math and gains on Hard Math too, so this is a disease of the instruct models. More extreme combinations exist: Qwen3-1.7B trained on Telugu drops 8.1pp on unseen English-content tasks; Qwen3-4B trained on Swahili drops 19.2pp on English, 14.8 on German, and 14.0 on Japanese. Performance in the training language itself is preserved, and multilingual training plus every other monolingual run stay clean.
The reasoning-language reward can trigger regressions too. gemma-3-1b-it on Swahili prompts with native-reasoning rewards drops sharply on Easy Math while the same setup with English rewards improves substantially; gemma-3-4b-it on Bengali behaves the same way; SmolLM3-3B on Chinese with native rewards regresses on Hard Math, with English rewards only mitigating. These effects are model-and-language specific: gemma-3-1b-it on Bengali shows no such regression, nor does gemma-3-4b-it on Swahili.
Three operational conclusions for non-English RLVR. Native-language reasoning costs little on most languages; for low-resource targets, reasoning in English is safer. Average gains hide regressions, so any multilingual RLVR run needs a broad evaluation panel covering many languages and task types, with English-content tasks and hard math as the two canaries. And monolingual language choice needs care, since specific combinations collapse specific capabilities in other languages; multilingual training that includes English prompts largely mitigates the Hard Math regressions.
The authors list theirs: the training data is procedurally generated and template-based, without the variability of real user queries; one shared recipe covers all configurations with no per-setting tuning, so some negative results might shrink under tuned hyperparameters; single seed throughout; and the scope stops at automatically verifiable math, logic, and puzzle tasks, leaving reward models, tool use, and open-ended tasks untouched. This reader adds one: the mechanism behind the regressions is never explained. Why Qwen3-instruct on Swahili precisely destroys unseen English-content tasks while the Base model is immune is documented, not understood. Eight of the nine models sit at or below 8B, so behavior at scale is unverified.