Testing 5 Models as Salesforce Agents: Silent Failures Fool Them All
TruthDeep8659 · reddit · 2026-10-28
The author ran Claude Opus/Sonnet/Haiku, GPT-6.1 Sol, and DeepSeek through 50 tasks (8 tools) inside Salesforce, graded on actual database changes rather than agent replies. Key findings:
- When the save tool returned "success" without saving, every model reported at least 15 of 20 such tasks as done;
- Models that re-checked a record after writing scored higher;
- Most failures came from implicit fields Salesforce changes on its own;
- Having write tools return the saved record would likely fix most failures.
A full write-up, code, and transcripts are linked.
Related event: Claude Opus tops Salesforce agent benchmark at 87%(3 posts)→
More from coding & agent
- Agents built a custom inference engine: AMD MI350X serves GLM 5.3 FP8 37% faster than NVIDIA B200 — yisongyue · 2026-10-28
- Kiro explains how 3 engineers and agents keep 1,000 PRs a week stable with 80+ automated checks — rseroter · 2026-10-28
- Single vs multi-agent: same task, 4 setups, 3x cost gap measured — latent_lurker8587 · 2026-10-28
- Mapping the stack that gives agents inboxes, phone numbers and wallets — Similar_Confusion534 · 2026-10-28
- Solo dev's plan to work around Meta's WhatsApp error 3441038 with a fresh number — Fit-Stretch-182 · 2026-10-28
- One prompt to a 30-second explainer video: 4 rules that make code-drawing agents reliable — _haiderfarooq · 2026-10-28