Zuse paper: decision models for 27 European languages hit 87.5% accuracy at 27B, lag English suites behind Jev and Kev
kalyan_kpl · x · 2026-11-05
An arXiv paper (2611.06238) introduces Zuse, a family of decision models (4B/9B/27B) for 27 European languages: instead of generating text, they read a document and return probabilities over a fixed option set.
Method: a low-rank adapter plus a pointer head trained on frozen open-weight bases. Prior 2026 decision models published no results outside English; Zuse fills that gap.
Results: on a European suite of 11 held-out sources, 9 task groups and 27 languages, the 27B model reaches 87.5% accuracy with pooled ECE 0.016 (0.035 per-source weighted). On English public suites it trails Jev and Kev.
Three findings:
- Miscalibration is structured by task, not language: knowledge tasks are overconfident and acceptability judgments underconfident at all sizes, so a single temperature cannot fix both
- The share of decisions automatable at a 5% error budget tracks chance-corrected skill (Spearman 0.97) more than raw accuracy (0.91)
- Reach beyond training languages is set by the base model: across 210 languages, changing the training run moved scores by at most 2 points within a size
More from Research
- Qwen3.5-0.8B 'weight-space programming' finetune trends on Hugging Face — RikiyaT · 2026-11-05
- Emergent misalignment published in Nature: narrow finetuning causes broad LLM misalignment — davidmanheim · 2026-11-05
- FineWeb-30M speedrun record falls to 14.2s on one 8xH100 node, a 1.64x speedup — RobertTLange · 2026-11-05
- National Geographic German edition features HPI's GorillaMonitor AI for gorilla conservation — gdm3000 · 2026-11-05
- Translucent: language agents for autonomous experimental engineering, code open — YashasGunderia · 2026-11-05
- Floridi et al. propose portable spec for agent harnesses, split into 8 components — mikeflache · 2026-11-05