Jack Rae on low-resource language eval and transfer efficiency
jack_w_rae · x · 2026-08-20
Jack Rae notes that evaluating models in low-resource languages like Welsh shows lower capability but uses a much smaller fraction of token FLOPs (e.g., 1/1,000th). He observed this in GPT-4 and indicates that transfer learning improves with scale.
Related event: Researchers Debate Weaker-Than-Expected Cross-Lingual Transfer in LLMs(5 posts)→
More from Research
- UW–Madison Hosts Brain Decoding Challenge for #MLM26 — Pseudomanifold · 2026-08-20
- Study: Social media feeds often clash with user values — mattgroh · 2026-08-20
- FrankenRedis: A memory-safe Rust reimplementation of Redis, built with AI agents — doodlestein · 2026-08-20
- IonNet Framework Predicts Ion Mobility Without Crystal Structures — bravo_abad · 2026-08-20
- DepthFirst releases dfbench for defensive cyber security evals — andreamichi · 2026-08-20
- IBM study: Rephrasing questions swings LLM benchmark scores by up to 74.7 points — rohanpaul_ai · 2026-08-20