Alibaba’s HSCodeComp benchmark finds top AI agents still far below human tariff experts
jiqizhixin · x · 2026-07-29
Alibaba Group introduces HSCodeComp, a benchmark that asks AI agents to infer precise 10-digit HS codes from product descriptions and tariff rules.
- The task requires layered reasoning from broad product categories down to narrow exceptions, closer to how a lawyer or doctor applies rules.
- Results are harsh: the best AI agent reaches only 46.8% accuracy, while human experts score 95.0%.
- Adding more reasoning steps can actually hurt, with the model drifting off track instead of improving.
- The paper frames HSCodeComp as a realistic, expert-level benchmark for hierarchical rule application.
More from Research
- Transforming quadruped robot runs on real hardware: per-pose RL policies in Isaac switched on morph — _Stocko_ · 2026-10-16
- KinoGen synthesizes robot loco-manipulation reference motions directly via kinematically grounded video generation — ChongZzZhang · 2026-10-16
- Science Robotics paper introduces F³, predictive event-based vision beyond optical flow — CSProfKGD · 2026-10-16
- VideoGen-Agent: An 8B Multimodal Agent Turns Video Generation Into a Tool-Use Loop — jiqizhixin · 2026-10-16
- Comprehension Audits: New Paper Proposes a Gate for AI-Written Code at Frontier Labs — ronbodkin · 2026-10-16
- Not Boring #215: Cammarata's Clean Networks Claim, Tesla's Flying Roadster, Tau Cut 73% — Not Boring (Packy McCormick) · 2026-10-16