"4 of 6 signals" vs "qualified": SpecJudge debates how to label recommendation confidence
jokiruiz · reddit · 2026-11-04
The author of SpecJudge, an open-source tool where a local LLM judges project specs to recommend models (every rating must cite a spec fragment), is soliciting design feedback on reporting confidence. Ground rules: no 0–100 scores (false precision) and no self-reported model confidence. Option A is an inspectable "4 of 6" signal count that hides equal weighting; Option B is an ordinal (solid/qualified/thin/nominal) with published components. The six signals: per-dimension citations, no unsupported dimensions, spread citations, hardest requirement explicitly stated, real work artifacts, and full context coverage.
Related event: SpecJudge Seeks Input on How LLM-as-Judge Should Report Confidence(2 posts)→
More from coding & agent
- awesome-ai-apps: 138 runnable LLM apps and agents, 16.7K stars and MIT-licensed — Arindam_1729 · 2026-11-04
- 130+ runnable AI app examples, 16.6K stars: awesome-ai-apps covers every agent framework — Arindam_1729 · 2026-11-04
- 6-step workflow: Grok Bot + Figma MCP turns text prompts into real frames and components — nima_owji · 2026-11-04
- 10 open-source repos that let AI agents see your screen and control computers and phones — victor_explore · 2026-11-04
- Devs debate retiring 'vibe coding': the term stuck around because coders tied ego to writing code — fekdaoui · 2026-11-04
- A month at NetBird: agents devs ignore the networking layer at their peril — victorialslocum · 2026-11-04