Goodfire's Predictive Data Debugging Forecasts Post-Training Behavior Changes Before Spending Compute
allen_ai · x · 2026-09-10
Ai2 and Goodfire detail a new post-training research result: teams usually discover unwanted behavior changes only after training finishes, then guess which of hundreds of thousands of examples caused them. Goodfire built "predictive data debugging" on Ai2's open stack — estimating which behaviors preference training will strengthen or suppress before committing compute. The open stack includes the Dolci preference dataset for Olmo 3, intermediate checkpoints with reproducible recipes, and OLMES evals for measuring capability shifts.
More from coding & agent
- Gen games don't need to be realtime — turn-based interactivity works, devs argue — djcows · 2026-10-27
- How to ace the "send us something cool" AI interview: research interviewers with agents, then focus — brandon_galang · 2026-10-27
- Give AI a clean personal-data-free machine with full control: agents running 24/7 teach you more than any course — EXM7777 · 2026-10-27
- 20 papers across 6 areas to master production-grade agentic AI engineering — MaryamMiradi · 2026-10-27
- LangChain SF meetup to compare decision model Jev vs LLM judges for agent evals — LangChain · 2026-10-27
- How Clang recognizes your popcount loop and swaps it for a single CPU instruction — jimmykoppel · 2026-10-27