Discussion on low-efficiency data transfer suggests a public transfer learning competition
willdepue · x · 2026-08-20
Will Depue discusses the efficiency of data transfer learning, suggesting that even between highly similar data (e.g., English and Welsh global news), efficiency might be low (1/10 or 1/100), and abysmal for vastly different domains (like Python and DNA). He proposes a public competition, similar to the nanogpt speedrun, to test transfer capabilities: using infinite Chinese data with fixed English data to minimize English PPL or English knowledge evals.
Related event: Researchers Debate Weaker-Than-Expected Cross-Lingual Transfer in LLMs(5 posts)→
More from Research
- LLMs predict forward reading, but fail to explain eye regressions — tallinzen · 2026-08-20
- CRUX Team Rebuttal: Scaffold Not the Bottleneck in AI Research Agents — random_walker · 2026-08-20
- GeneralistAI demos GEN-1.5 robot capable of one-shot learning — teortaxesTex · 2026-08-20
- FRONT 3.1 Paper: Cognitive architecture with "Digital Somatic Body" and homeostatic drives — Sufficient-War4616 · 2026-08-20
- Small CNNs can reliably detect image generator sources via spectral signatures — kwangmoo_yi · 2026-08-20
- AI Analysis of Microplastics Health Impact Shows Little Evidence — juliey4 · 2026-08-20