PolyRoute: Intel releases 12-language code-mixed routing benchmark, best fine-tune hits 96.6%
Mohammed Nayeem Khan · hf · 2026-11-12
Intel releases PolyRoute, a multilingual benchmark for typed-decision routing: 322 native-language support messages across 12 languages and 9 writing systems, 156 code-mixed or romanised messages (Hinglish, Spanglish, Arabizi), 967 optional native training rows, and a three-way department-routing protocol.
Quality checks: labels come from Qwen3.8-27B with a blind second pass (91% kept); an independent model, gemma4-31b, agrees on 97.2% of test items; max character 5-gram similarity between test and training text is 0.33.
A 13-system leaderboard: English-root checkpoint 61.2%, shipped multilingual checkpoint 77.3%, best fine-tune 96.6%; TF-IDF + logistic regression reaches 91.9%. On four withheld languages the multilingual systems score 77%/90%/93%; on code-mixed text the English root (85%) nominally leads the multilingual checkpoint (79%; p=0.134). The split only resolves differences above 4 points. All texts and labels are LLM-generated.
More from Research
- ICLR abstract word-frequency shift: 'novel' and 'comprehensive' plummet as LLM fingerprints fade — y_m_asano · 2026-11-12
- Evolving Infra: agent-adaptive LLM serving cuts latency up to 4.1x vs NVIDIA Dynamo — mit-han-lab · 2026-11-12
- Isomorphic Labs President on Using AI to Turn Drug Discovery into Precision Engineering — No Priors · 2026-11-12
- CreativityPrism benchmark scores 17 frontier and open LLMs across 8 creative tasks — DanielKhashabi · 2026-11-12
- KQuant: open-source MLX quants cut Qwen KL divergence up to 3.7x vs public 4-bit — m4rkGS · 2026-11-12
- Possible eval leak: all frontier models fail a GPQA Diamond question whose official answer seems wrong — qberthet · 2026-11-12