Model Speed Depends Heavily on the Task: Wall-Clock Time Across Benchmarks
rickasaurus · x · 2026-11-13
A thread by Adam Holtererer compares wall-clock time per task for various models across benchmarks. The core point: how fast a model feels depends largely on what kind of task you give it, with significant variance across task types.
More from Models
- Startup Navi scores 73.3% on OfficeQA Pro, beating frontier labs on same model — ycombinator · 2026-11-13
- Claude account folklore in China: KYC prompts mean low weight, environment decides risk — jiayuan_jy · 2026-11-13
- OpenAI's $0.11 model beats the priciest LLMs in new Slack agent benchmark — heyshrutimishra · 2026-11-13
- Dean Ball: Open-ended long-horizon domains hit a wall near human level — verifiability is the crux — deanwball · 2026-11-13
- JevBench Splits Open-Weights Models and Hosted APIs onto Separate Leaderboards — airesearch12 · 2026-11-13
- JevBench update: top three within cost and speed caps are all open weights — airesearch12 · 2026-11-13