Camellia benchmark measures LLM cultural biases across 9 Asian languages and 6 cultures
Tanmoy_Chak · x · 2026-10-26
- Researchers released Camellia (arXiv 2025), an open-source benchmark (MIT-licensed GitHub repo) for measuring entity-centric cultural biases in LLMs for Asian languages.
- It covers 9 Asian languages (Chinese, Japanese, Korean, Vietnamese, Urdu, Hindi, Gujarati, Marathi, Malayalam) and 6 Asian cultures (Chinese, Japanese, Korean, Vietnamese, Pakistani, Indian).
- The dataset ships two parts: annotated cultural entities across 6 entity types (tagged to Asian or Western culture), and masked contexts including camellia-grounded culturally-grounded contexts.
- The benchmark addresses the Western/English skew of existing bias benchmarks.
More from Research
- ETH Zurich paper: dynamic evaluation allocation matches 10x-budget model rankings — zouharvi · 2026-10-26
- DuoLM: Fast LLM-Based Personalized Recommendation via Compact Encoder-Ranker — GoogleDeepMind · 2026-10-26
- DISC Learns to Combine Time-Series Dataset Similarity Metrics for Transfer Prediction — Yen-Ku Liu · 2026-10-26
- Adobe's SliderEdit++ Improves Slider-Based Image Editing by 18.5% with New SlideBench — Arman Zarei · 2026-10-26
- DeepMind Builds Auditable AI Text Detectors with Sparse Autoencoders — GoogleDeepMind · 2026-10-26
- CoRE Framework Learns Rule Routing for Knowledge Retrieval Without Supervision — Zhisheng Qi · 2026-10-26