SpecJudge Seeks Input on How LLM-as-Judge Should Report Confidence

SpecJudge, an open-source tool that uses a local LLM as a judge to recommend models based on project specs, is publicly soliciting feedback on how to report recommendation confidence, with its author rejecting 0-100 scores and debating formats like "4 of 6 signals met" versus "qualified."

2026-11-04 ~ 2026-11-04 · 2 related posts

• "4 of 6 signals" vs "qualified": SpecJudge debates how to label recommendation confidence — jokiruiz · 2026-11-04
• SpecJudge author argues LLM-as-judge tools should never ask models for self-reported confidence — jokiruiz · 2026-11-04