A research report isn't a completed research task: rethink how we judge AI science agents

Remind_me_to_Learn · reddit · 2026-09-07

The author argues we judge AI research systems too early: finding papers and producing a cited report is where real scientific work begins, not ends.

For studies built on spreadsheets or datasets, someone still must inspect raw tables, clean and align variables, choose a defensible method, run analyses, generate figures, and trace conclusions back to the data — and handle mid-task corrections without redoing everything.
The bottleneck has shifted: can the system maintain task state, work in real file/code environments, recover from failures, and leave inspectable artifacts?

The author demos Apodex, which in a Deep Discover example (EASIX vs. overall survival after EBV reactivation in allogeneic transplant patients) audits data, cleans it, picks a statistical method, and outputs Kaplan-Meier curves and tables — preserving unaffected work when criteria change mid-task.

Closing question: should "done" mean a well-cited report, or a reproducible package of sources, cleaned data, steps, figures, and limitations?

Original post →

A research report isn't a completed research task: rethink how we judge AI science agents

More from coding & agent