JevEval uses Jev as the judge for custom LLM evals, with DeepEvent handling scoring
solyarisoftware · x · 2026-10-14
JevEval (v0.1.0, Apache-2.0) is an open-source approach to custom LLM evals that uses Jev as the judge instead of another generative LLM. You pre-define exact evaluation decisions (yes/no, score, or choice-based, with weights), Jev returns probability-based decisions, and DeepEval computes the final score — keeping scoring logic outside the model. Useful for LLM apps, agents, and RAG pipelines, e.g. checking whether an agent picked the right tools, followed the expected action sequence, and completed the task.
More from coding & agent
- Anthropic code merges up 12x in 4 months; 7 engineers build new source control with Claude — bittingthembits · 2026-10-14
- Flattened structs on 64-byte cache lines: the simple principle of high-performance code — ryunuck · 2026-10-14
- Deck launches as an always-on AI product manager you trigger from your agent — Scobleizer · 2026-10-14
- Claude ports Mario Odyssey to macOS overnight after learning from an Animal Crossing port — dotey · 2026-10-14
- Gemini 4 Argon goes GA in GitHub Copilot, beating prior Gemini models on coding — 0xkarasy · 2026-10-14
- Gemini 4 Argon lands in Cursor: 52.8% on CursorBench, matching Opus 5.5 at $2/$10 — gaganghotra_ · 2026-10-14