Shopify & Toloka build ground-truth flywheel boosting agent accuracy to 89%
MParakhin · x · 2026-08-20
Shopify partnered with Toloka to build a "ground-truth flywheel" for their AI commerce agent, Sidekick. By incorporating expert human feedback to align the LLM judge, the mechanism improved judge accuracy from 75% to 89% in one month, fed corrected trajectories directly into SFT & RL, and halved per-task annotation costs.
More from coding & agent
- Refuse cross-session data sharing in Claude settings — dotey · 2026-08-21
- Ramp Enables Agent Payments: AI Agents Can Now Spend with Audit Trails — kleffew94 · 2026-08-21
- Leaked Stripe Letter Reveals OpenRouter Acquisition, Calls Agents 'Economic Actors' — rohanpaul_ai · 2026-08-21
- Slack launches Slack Code to integrate Claude, Devin, and Copilot into team dev workflows — tristanbob · 2026-08-21
- Chroma Launches Foundation: Self-Improving Memory for Agents — nbaschez · 2026-08-21
- Epho: Run Claude Code and other coding agents in the cloud via API — karakanb · 2026-08-21