Copy-Paste Language Models boost CoT, long-context and RAG accuracy at under 3% overhead
yoavartzi · x · 2026-10-17
- A new paper introduces Copy-Paste Language Models (CPLMs), which learn to copy tokens better.
- Gains: improved downstream accuracy, chain-of-thought performance, long-context performance, and memory efficiency for RAG — all at under 3% overhead.
- CPLMs also (partially) mitigate the LM head bottleneck. Paper link in the original post; John Langford called it "quite nice."
Related event: Cornell's Copy-Paste Language Models Boost Copying with Tiny Overhead(3 posts)→
More from Research
- A 'pre-numeric' AI math layer trains patterns before numbers exist — debreuil · 2026-10-17
- research-rag: local CPU-only PDF/ebook search with browser, CLI and MCP access — kishkisan · 2026-10-17
- Epoch AI's SpireClimb: models beat human author at Slay the Spire but still err each act — Jsevillamol · 2026-10-17
- Satya Nadella: Microsoft research uses ML to extend weather forecasts 2-6 weeks out — satyanadella · 2026-10-17
- SAE features are only partially compositional, new paper finds — OpenSAE · 2026-10-17
- Claude Mythos 5 designs quantum circuits beating published best, up to 10x gains — Distinct-Question-16 · 2026-10-17