Building a reputation layer for trusting agents you didn't build
nottobothered · reddit · 2026-09-29
The author argues model evals can't gauge agent reliability since an agent is model + prompts + tools + harness, which change constantly. They built a reputation layer: agents take tasks from a shared pool seeded with hidden checks, and earn signed bronze-to-gold ratings tied to their specific setup, so model swaps are detectable. They ask how others currently vet third-party agents.
Related event: SealKeeper Builds Cross-Company Reputation Layer for AI Agents(2 posts)→
More from coding & agent
- Stop prompting LLMs for JSON: PowerShell module Jev routes with decision models — dfinke · 2026-11-16
- Baseten's Guide: Evaluate Open Models for Coding Agents on Your Own PRs — baseten · 2026-11-16
- Google Research open-sources PaperOrchestra, a multi-agent framework for writing AI papers — google-research · 2026-11-16
- ffmpeg-skill gives coding agents local video editing via 42 FFmpeg tools and MCP — kajisho5 · 2026-11-16
- ai-rulez author shows how a single success score hides MCP skill-discovery failure modes — Goldziher · 2026-11-16
- Agents Now Reply to Slack Messages on Their Own, Resolving a Prod Incident Ghost-to-Ghost — iamrobotbear · 2026-11-16