Training Kimi on 1,465 PDF tasks more than doubles general task success
echen · x · 2026-09-30
New research asks: if you train on the underlying skill behind the widely used GDP.pdf eval, does the model just get better at PDFs — or learn something more fundamental?
The team post-trained Kimi K2.7 on 1,465 GDP.pdf companion tasks:
- GDP.pdf improved from 11.0% to 24.5%;
- On GDPval (tasks with no PDFs, in an environment with files, tools, spreadsheets and deliverables), full-task success more than doubled and 90% rubric pass rate rose +13.2pp.
Why did it transfer? Trajectory analysis shows that before training, Kimi often grabbed the first plausible source and started working; after training it looks around first — what's here, what matters, is anything missing? In one task, the base model opened the first weekly timesheet and prematurely computed a monthly report; the trained model correctly gathered all four weeks first.
The framing: a PDF is a small evidence environment — a good professional PDF forces you to figure out where facts live, which charts matter, and whether evidence suffices. Training on rich PDFs taught the model to gather all evidence before acting, a habit that persisted beyond PDFs.
Related event: Kimi Research: Training on 1,465 PDF Tasks Doubles General Capability(2 posts)→
More from Models
- Stealth Model 'silicon-2' Surfaces in Devin, Suspected to Be Anthropic's Next Model — kimmonismus · 2026-11-16
- Google hides Gemini model label in AI Mode, rolls Gemini 4 Carbon to Pro subscribers — gaganghotra_ · 2026-11-16
- Open-weight Karpo-Decision-31B tops Decision Index at 73.9, one forward pass per answer — CodeByPoonam · 2026-11-16
- Nathan Lambert calls out Ethan Mollick for 'selectively ignoring' Kimi in model comparisons — natolambert · 2026-11-16
- Eval Firm Trajectory Told 40 Agents "NEVER Stop" in Harness Without Telling Epoch — peterwildeford · 2026-11-16
- Unsloth Q8 crushes default Q3 in a self-hosted test over 331 financial PDFs — No_Cartographer3953 · 2026-11-16