Pleias releases TD Bench, a controlled multilingual benchmark of fluency judge models across 25 languages
antoine_chaffin · x · 2026-10-23
Pleias' new paper, "Which Model Should Judge That Language?", tackles LLM-as-judge selection for multilingual fluency: judges score text 0-5 for naturalness to a native speaker, but competence, cost, and latency vary, especially for low-resource languages — do you need a frontier model, and is even that enough?
A pilot on grammatical error correction showed wide variation in how much teachers edit broken texts across languages. The team therefore built the Translationese Detection Benchmark (TD Bench): original-language web texts plus mild, medium, and heavy synthetic translationese degradations, where a judge succeeds by scoring source text above its degraded version. Evaluated across 25 languages by language and severity to help readers pick a judge per target language and fluency level.
More from Research
- BrickWright: agents build LEGO assemblies with perfect physical validity — cmu-gil · 2026-10-24
- HyperDrone: ray-tracing drone simulator runs at millions of FPS for sim2real — rl-tools · 2026-10-24
- A byte-level transformer works, but BPE exists for a reason: context length — max_paperclips · 2026-10-24
- Community note debunks claim that distant tokens never share MoE experts — NathanpmYoung · 2026-10-24
- Many LLM users never touched classic classifiers — old-school ML hygiene is fading — antoine_chaffin · 2026-10-24
- Google shows neural cellular automata can do multi-step reasoning on decentralized hardware — tyrell_turing · 2026-10-24