PACT framework finds LLM cultural-norm behavior shifts 7.8% by country, vs 1% age, 0.7% gender
radamihalcea · x · 2026-11-02
Rada Mihalcea's team presented an EMNLP 2026 main-conference oral paper, "Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models."
- Introduces PACT (Personal-Preference and Cultural-Norm Trade-off), a framework testing whether LLMs follow cultural norms or honor personal preferences in social decisions.
- Key finding: model behavior shifts more with country context (7.8%) than age (1%) or gender (0.7%), and shifts non-uniformly after instruction tuning.
- A five-country human study shows culture-following is driven mainly by scenario country, with lowest agreement when people judge their own culture—revealing within-culture pluralism.
- Human-LLM alignment: models match majority choices but fail to capture response distributions and uncertainty (best correlation only 0.24).
Takeaway: alignment evaluations should go beyond majority answers to capture cultural pluralism and disagreement.
More from AGI Musings
- DeepSeek researcher: 'Don't make any mistakes' is the underrated moat — Yuchenj_UW · 2026-11-02
- Mathematician: LLMs matter for massive proofs, not short insight papers — littmath · 2026-11-02
- Martin Ford vs Pedro Domingos debate whether AI will take most of our jobs — MFordFuture · 2026-11-02
- Ethan Mollick: GPT-4 alone would have had huge impact, just slower adoption — emollick · 2026-11-02
- Hamel Husain: AI benchmark companies are in the same business as credit rating agencies — HamelHusain · 2026-11-02
- Karpathy defines slop as "regretted attention"; idea: weekly self-labeling to train a personal AI filter — kellerjordan0 · 2026-11-02