Repetition Mismatch paper shows why data mixture experiments don't scale at EMNLP
MarekRei · x · 2026-09-05
- The paper "Repetition Mismatch: Why Data Mixture Experiments Don't Scale and How to Fix Them" has been accepted to EMNLP2026 in Budapest.
- It investigates how dataset repetitions affect optimal data mixing in data-constrained training settings, showing that mixture experiment findings break down at scale due to repeated data, and proposes fixes.
- The author shared a thread with details.
More from Research
- New tendon-based robotic hand designs trained with PPO to learn one-armed catch — _krishna_murthy · 2026-10-23
- Google QuantumAI and Kaggle launch $150K verifiable quantum circuit compilation competition — JFPuget · 2026-10-23
- Diffusion LLM bias rankings shift with sampling configs, persist after debiasing — Bollegala · 2026-10-23
- Aggregating lab tokenizers reveals ~994 tokens stable across Google, DeepSeek, Meta, Anthropic — kalomaze · 2026-10-23
- Researcher uses Grok to hunt LLM-driven UAE bot network on X with self-written prompts — FlorianGallwitz · 2026-10-23
- Dev converts Qwen3-0.6B into a MoE model: quality drops and no speed gain at this scale — Bayliner1980 · 2026-10-23