Tracing tool-call vs tool-output costs separately in LangChain with Arize Phoenix
Normal-Blueberry-385 · reddit · 2026-09-03
A developer shares an observability setup for LangChain apps using Arize Phoenix + OpenTelemetry, arguing four metrics must never be collapsed: LLM tool-call generation tokens/cost, tool execution (name, args, latency, output size, provider cost), tool-output consumption as next-request input tokens, and the final LLM response. Implementation uses a custom BaseCallbackHandler with a unique tool.call.id to correlate parent/child spans (LLM → tool → next LLM), plus centralized pricing config, sensitive-argument masking, and fail-safe tracing that never breaks agent execution. A worked example shows a tool's 850 output tokens measured separately from the 1,478 input tokens of the following LLM call.
More from coding & agent
- Free n8n workflow auto-pulls weekly ad reports across all platforms via Databox MCP — aftahi_ai · 2026-09-03
- 用 TRL+OpenEnv 开源复现「写代码画水彩」模型全流程 — huggingface · 2026-09-03
- Developers Praise Claude Design: Great Results with a Design System — KlausCodes · 2026-09-03
- Anthropic engineer on building self-improving AI systems: loops and graphs explained — goyalshaliniuk · 2026-09-03
- Wes Roth Builds Four Full AI Games on Claude Fable 5.1's Low-Effort Setting — Wes Roth · 2026-09-03
- Dev swaps in Cursor and Grok 4.6 subagents when hitting Claude Code limits to save cost — rudrank · 2026-09-03