How do you let SMEs safely edit and evaluate content feeding a RAG pipeline?
emmas__eye · reddit · 2026-10-29
A Reddit user describes their team's agents that answer questions over structured clinical data, supported by a RAG system of short guidance snippets mapping clinical concepts to their data model.
Domain experts (clinicians, research scientists) should own this content, but it lives in the repo — and moving it out raises unvalidated-edit concerns. They want: a friendly SME editing UI with version history, review, and rollback; automated evaluation of edits before going live (evals already exist via CLI/endpoint); and a way to find snippets not covered by test cases.
They note KM tools handle editing/approval but not content evals, eval tools don't handle SME editing, and PromptLayer targets prompts rather than content. They ask whether to build or buy and how others gate SME edits.
More from coding & agent
- Extract vs Load: comparing the same knowledge graph built two ways, edge by edge — adnan_hashmi · 2026-10-30
- A signup is a saved user record: count the database write, not the click — richardreis · 2026-10-30
- Mobilerun lets AI agents control a real iPhone from the cloud in under 2 minutes — Scobleizer · 2026-10-30
- Colloquium: Nathan Lambert's markdown-native slides tool built for agents — May_F1_ · 2026-10-30
- AI workflows have no playbook: what worked two weeks ago is already outdated — jacalulu · 2026-10-30
- Every deleted 86,730 lines of its personal-agent product, rebuilt as one shared company agent — every · 2026-10-30