I graded 32 marketplace AI agents — the F's all failed the same way

xrplincoln · reddit · 2026-11-17

A developer built an independent grading system that assigns realistic tasks to 32 agents from a public marketplace and scores them with independent judges. Results: 6 A, 7 B, 8 C, 5 D, 6 F.

Two things separated A from F:

The author suggests two cheap checks for agent evals: one question with no knowable answer, and a hard answer-length cap. Every agent gets a public report with all tasks, answers, and scores.

Original post →

More from coding & agent

coding & agent channel →