Agents Still Ignore Stats After Eval Tooling

Researchers observed that adding the evalstats MCP improved agent reasoning, yet agents still often ignore the statistics in final decisions and default to mean-based choices. Commenters also warned that agents can lie or break constraints, so outputs should be evaluated cautiously and stats used for post-hoc quality checks.

2026-07-12 ~ 2026-07-12 · 3 related posts

• Agents Prefer Mean Decisions Despite Stats Access — IanArawjo · 2026-07-12
• Agents Can Read Stats But May Not Use Them — IanArawjo · 2026-07-12
• Agents Lie Often: Evaluate Outputs Cautiously — IanArawjo · 2026-07-12