Adaption Labs: agent-written checklists filter training data, boosting win rates 9.4-13.2%
sarahookr · x · 2026-10-09
Adaption Labs (Sara Hooker, Wei-Yin Ko) published new research on agentic checklist generation for automated data quality control, targeting the 44 expert domains (medicine, law, business, etc.) where there is no single correct answer to check against.
- Method: A Discovery Agent reviews past runs of AutoScientist, the company's automated training system, then autonomously writes and updates domain-specific yes/no checklist items—replacing the vague "is this example good?" with verifiable questions—to filter low-quality data before training.
- Results: Models trained on 10,000 checklist-filtered examples had relative win rates 9.4%–13.2% higher than those trained on 10,000 randomly sampled examples, holding across general, medical, legal, and technology domains.
- Significance: Checklists aren't new; the novelty is that the list is written and updated by an agent with minimal human input, making automated data QC possible in otherwise unverifiable domains.
Related event: Adaptation AI: Agentic Data Checklists Deliver Consistent Training Gains(3 posts)→
More from Research
- Why AI doesn't actually read words: from BPE subwords to byte-level models like BLT and H-Net — jbhuang0604 · 2026-10-09
- NVIDIA details HSTU recommender inference stack with up to 5.93x lower latency — PyTorch · 2026-10-09
- Preprint: LLMs store numbers as curves and helices, but compute comparisons differently — tweetsatpreet · 2026-10-09
- Best model was cheapest: open-weights model ran 669 clinical decisions for 1.7 cents — antoine_chaffin · 2026-10-09
- Ofir Press Points to ExcelBench as the Right Scale for Benchmarking Coding Agents — OfirPress · 2026-10-09
- tangermeme, a genomic sequence-to-function modeling toolkit, published in Nature Methods — anshulkundaje · 2026-10-09