AutoRubric 1.7.0 grades images against rubrics, prompt caching cuts per-criterion cost ~25%

deliprao · x · 2026-11-11

Delip Rao shipped AutoRubric 1.7.0 and detailed it in the inaugural post of a new blog:

Image inputs: submissions can now hold images, letting rubrics grade charts, screenshots and scanned pages (e.g. checking axis labels), scored by both LLM judges and decision models.
Cost trick: per-criterion prompts are reordered so the provider's prompt cache reuses the submission across an item's calls — per-criterion grading cost fell by nearly a quarter for two of three judges.
Other changes: the meta-judge is told it's reading a rubric, and misconfigured or out-of-credit judges now raise errors instead of abstaining.

Includes an Alice-vs-Bob chart-grading example. Install with pip install -U autorubric.

More from coding & agent