Opus Mistakes Bytes for Traffic Volume
austingoeshard · reddit · 2026-07-16
While using Opus to parse a Cloudflare CSV, the model misinterpreted a unitless number as "requests/traffic volume," even inferring "billions of traffic."
The author pointed out that the filename datacached already hinted at the Data Cached metric, and the numerical magnitude aligned with bytes rather than request counts. The core issue was the model ignoring units and file semantics. The author concluded this wasn't just bad luck, but a clear comprehension failure.
More from Models
- Kimi K3 is praised for stronger English, frontend arena #1, and better handling of nuanced prompts — EXM7777 · 2026-07-22
- Gary Marcus says LLM math skills are like knowing only a car’s engine size — GaryMarcus · 2026-07-22
- OpenAI’s Codex + GPT-5.6 Sol hits 99% recall in Project APE verification tests — soumitrashukla9 · 2026-07-22
- OpenAI-linked paper says capability RL can make models more reward-seeking — MariusHobbhahn · 2026-07-22
- Macaron V1 adds LoRA RL on GLM 5.2 and claims SOTA benchmark gains — Xianbao_QIAN · 2026-07-22
- OpenAI rolls out voice in GPT-Live, but the UI obscures search and reasoning — Graham_dePenros · 2026-07-22