LangSmith Tuned Evaluators Cut Costs by 82% vs. Frontier Models
hwchase17 · x · 2026-08-19
LangChain launched LangSmith Tuned Evaluators, starting with Perceived Error. This tool runs on production traces to catch undesirable agent behavior and attach feedback for improvement processes.
Key features:
- Holistic Scoring: Evaluates entire conversations rather than individual steps.
- Cost Efficiency: In benchmarks, the tuned model achieved higher accuracy at 82% lower cost than frontier models.
- Dev Workflow: Directly integrates feedback into agent improvement cycles, easing the burden on developers.
More from coding & agent
- atuin v18.24 adds cross-device sync to resume AI threads across Claude and Codex — braelyn_ai · 2026-10-16
- Agents turn conservative as context grows: edits shrink from 150 to 30 lines — dbreunig · 2026-10-16
- ai& becomes an official built-in Cline provider with 13 open-weight models — DavidBennett__ · 2026-10-16
- Open-source rizin-mcp lets LLMs run local binary and malware analysis via Rizin — Pure_Emergency1424 · 2026-10-16
- x402 as a free GTM channel: agents discover 100+ tools, no API keys needed — 0xJeff · 2026-10-16
- Sam Altman: stop getting better at prompts, build loops and graphs that prompt for you — goyalshaliniuk · 2026-10-16