Cohere Labs' CulturalRiddles: top AI model hits 79%, some cultures just 22%
Cohere_Labs · x · 2026-10-26
Cohere Labs released CulturalRiddles, testing 28 AI models on human-written cultural riddles from 167 writers across 45 countries and 51 languages (6,200 riddles).
Key findings
- Best model reaches 79% accuracy overall, but drops to 22% or lower for some cultural communities
- Translation boosts average accuracy from 22% to 32%
- 31% of reviewed answers were fabricated, and models rarely admit uncertainty
Takeaway: speaking a language isn't the same as understanding its culture. Artifacts are open-sourced.
More from Research
- Alibaba's four-generation TTS evolution: CosyVoice to Qwen-Audio-3.0-TTS — solyarisoftware · 2026-10-26
- LoopUno: 52-Minute LoRA Training Makes 3B LLM Draft 2 Tokens per Step Losslessly — LucaAmb · 2026-10-26
- Author publicly seeks emergency ICLR reviewers for three papers — _onionesque · 2026-10-26
- Cambridge NLIP brings 8 papers to EMNLP 2026, from tokenization to LLM agents — vlachos_nlp · 2026-10-26
- Context Bombs: deception tricks make attacking AI agents 23× costlier, zero success — tracebit · 2026-10-26
- V-Rubrics and EgoTools accepted to EMNLP 2026: rubric-based RL and egocentric tool reasoning — liuziwei7 · 2026-10-26