Researcher Says TypeSafe's Jev Mirrors His Autorubric LLM Eval Framework From 8 Months Ago
deliprao · x · 2026-09-21
Delip Rao says @typesafeai's new Jev product uses an abstraction strikingly similar to Autorubric, his open-source rubric-based LLM evaluation framework released 8 months ago and to be presented at COLM next month — with Jev's Noul/Score/Choice criterion types mapping 1:1 to Autorubric's. Autorubric (arXiv:2603.00077) unifies rubric-based LLM judging for non-verifiable tasks with bias mitigation, calibration, abstention, and the CHARM-100 benchmark.
More from Research
- minpts-check: one-run predictor tells if your HDBSCAN clusters survive min_samples changes — leland_mcinnes · 2026-11-12
- Architecture study sparks monitoring concerns over a multi-hop planning spike — ChowdhuryNeil · 2026-11-12
- SGLang Megakernel Pushes Muse Spark to 1,109 tok/s, 2.1x Over Baseline — hsu_byron · 2026-11-12
- ArcGrav: a cross-model Agent Control Language to cut token overhead — ArchitecturalGravity · 2026-11-12
- Open-source dual-arm robot solves tangram puzzles with 90 demos, ~60% success — DanGrover · 2026-11-12
- FutureHouse's deslopper trains Kosmos to write indistinguishably from human experts — proteinrosh · 2026-11-12