CritPt, SciCode and Terminal-Bench Science: three science benchmarks as RL environments
geoffwolfe · x · 2026-10-25
A comparison of three science benchmarks representing different evaluation paradigms: CritPt poses research-level physics problems over multi-turn context-stacking with one final verdict — the deepest yardstick available. SciCode chains advanced scientific computations where each solution feeds the next, with a reward at every link — the cleanest learning signal per dollar. Terminal-Bench Science drops an agent into a real shell for a long run with tools and retries, checked only at the end — the closest analogue to how models will actually do science. The author notes all three can become RL environments by wrapping each verifier as a reward, each with its own depth, horizon and reward rhythm.
More from coding & agent
- CASPA summit: chip-to-chip interconnect eats 60-70% of package energy, AI chip design targets 9-month cycles — ai · 2026-10-25
- Engineer orchestrates agents from his phone by voice: 'engineering has never been easier' — KevinNaughtonJr · 2026-10-25
- Automating insurance billing, AI consultant defeated by fax machines being too slow — gregmushen · 2026-10-25
- Tinker's StableReinforce reverse-engineered: it's REINFORCE plus score centering — andersonbcdefg · 2026-10-25
- $45K-prize Gemma build day at AGI House as Google opens Android AI integration — salgar · 2026-10-25
- Where does a workflow actually need an agent to pick the next tool? Reddit debates the boundary — ZinchaDEV · 2026-10-25