smevals: A Lightweight Eval Harness You Can Run with uvx
JeremyCMorgan · x · 2026-08-06
smevals is a lightweight eval harness that can be run entirely with uvx. It supports defining tasks in YAML, running across multiple models, separating grading from execution, and generating static HTML reports. The tool aims to provide a sane eval vocabulary without platform lock-in, making it easy to build and run eval suites.
More from coding & agent
- Specula: TLA+ Tool Automates Formal Specs, Finds Hundreds of Bugs — tianyin_xu · 2026-08-06
- Building Local AI NPC Systems with Emotion and Memory for Video Games — Patryk_Grzegorek · 2026-08-06
- ChipAgents Explores AI-Driven Formal Verification in Semiconductor Design — WilliamWangNLP · 2026-08-06
- Prime Agent achieves Turing-complete tool calling via single iPython tool — TheZachMueller · 2026-08-06
- Meta launches Muse Code, an AI agent for large code bases — TechCrunch AI · 2026-08-06
- Stop Reviewing Your Agent's Plan: Best Practices for AI Workflows — msg · 2026-08-06