SAP’s TRACE preserves tool knowledge and reaches 86% recall with greedy decoding
SAP · hf · 2026-07-28
SAP’s TRACE keeps tool knowledge while cutting retrieval latency
TRACE is a two-stage curriculum for enterprise LLM tool retrieval that aims to solve two problems in parametric retrieval: training can erase tool knowledge, and constrained beam-search decoding is too slow for real-time use.
The method first reuses ToolSense-style memorization SFT with LoRA to seed tool knowledge. Then it trains the model to emit a reasoning trace before producing a JSON list of tool tokens, using both ToolSense RRB pairs and expert-curated business-rule queries augmented with reasoning traces.
Reported results:
- Stage 2 preserves Stage 1 MCQ and QA probing accuracy
- MCQ accuracy improves by +3.2 pp
- QA probing improves by +9 pp
- On an enterprise catalog with 8,300+ tools across two product lines, TRACE reaches about 86% recall on Domain A and 60% on Domain B
- These retrieval numbers beat embedding baselines of roughly 27% and 52%, while using single-beam greedy decoding at production latency
The paper’s core claim is that TRACE makes parametric tool retrieval directly deployable without the beam-search slowdown.
More from coding & agent
- Demo shows an Android app workflow using Antigravity, BigQuery, and Cloud Run — rseroter · 2026-07-28
- Developer Uses Kimi Model to Uncover Linux Kernel-Level TLB Race Condition Bug — jd_pressman · 2026-07-28
- A small developer device adds support for eight coding agents — dee_hw · 2026-07-28
- SWE-rebench expands to five languages and adds local Qwen models — Fabulous_Pollution10 · 2026-07-28
- This Claude Code skill searches 2,039 Hormozi transcripts locally with timestamped citations — jacob_posel · 2026-07-28
- Dev Uses AI to Auto-Announce MRR, Ditching Stripe Dashboard Refreshes — KevinNaughtonJr · 2026-07-28