What's missing in LangSmith/Braintrust: tracing shows what agents did, not if it worked
Ok-Machine7983 · reddit · 2026-11-18
A team using Langfuse finds a key gap in LLM observability: traces show what an agent did, not whether it actually worked — e.g. did the record land in the CRM, was the refund amount right? They built custom checks on the side and ask how others handle this beyond tracing + evals.
Related event: Agent Observability Gap: Traces Show What Happened, Not Whether It Worked(2 posts)→
More from coding & agent
- Cognition launches Devin Bots: assign outcomes, not tasks, and agents run for weeks — marvinvonhagen · 2026-11-18
- FastMCP 4.2 ships Skills over MCP — drag a skill folder, get an MCP server instantly — HanchungLee · 2026-11-18
- KDD paper: naive 8B LLM critic drags AutoML search below random search, up to 0.245 AUC lost — rohanpaul_ai · 2026-11-18
- Amazon paper: weak LLM critics can sink search loops, ideal gain only 0.005 AUC — rohanpaul_ai · 2026-11-18
- OpenAI engineers secretly use Astra for Law to write PRDs and design docs — sherwinwu · 2026-11-18
- START-HERE.md: Mapping Your Vault So Agents Find Files Without Burning Tokens — tomcrawshaw01 · 2026-11-18