Harvard/Stanford Paper Proposes Four-Field Reporting Standard for Knowledge Work Benchmarks

The paper "Designing Benchmarks for Knowledge Work" from Harvard and Stanford proposes a four-field reporting standard clarifying what a benchmark measures and what its metrics mean. Drawing on CRMArena-Pro, the author stresses that fixing one bug does not equal completing a full workflow, calling for clearer coverage boundaries in agent evaluations.

2026-09-03 ~ 2026-09-03 · 2 related posts

• Harvard/Stanford paper proposes a 4-field reporting standard for knowledge-work benchmarks — Shahules786 · 2026-09-03
• Benchmarks should publish coverage boundaries: a coding agent fixing a bug isn't the whole job — Shahules786 · 2026-09-03