ATEST: load-aware test-time scaling cuts LLM inference energy up to 48x
Mobina Kashaniyan · hf · 2026-10-22
An Iowa State paper introduces ATEST, a resource-aware test-time scaling framework. Fixed sampling budgets ignore query difficulty and varying GPU costs — the same 4-sample query can use up to 4.1x more energy under GPU contention.
- ATEST-Online (default): no offline calibration; generates samples incrementally, checks plurality-answer stability, monitors live GPU load, and learns per-step energy cost online to decide when to stop.
- ATEST-Calibrated: picks per-query budgets from offline accuracy/cost models, suited to stable loads.
Evaluated on four reasoning/QA datasets with Phi-3-mini on an A100, with transfer tests on L40S and Qwen2.5-3B. Under high load on GSM8K, ATEST-Online hit 60.0% accuracy with zero deferrals vs 44.0% and 42.4% deferrals for Calibrated. Including offline profiling, Online used 42x less GPU time and 48x less energy, at the cost of higher tail latency and lower throughput on short tasks.
More from Infra
- Railway breach may have exposed users' decrypted env vars, all deployments paused — devenbhooshan · 2026-10-22
- Abacus.AI CEO beta-tests GPU-hosted apps on SuperComputer, runs 3D flying-car game — bindureddy · 2026-10-22
- Seroter's daily list: OpenAI models land on Google Cloud, PHP CVEs, Genkit Go 1.14 — rseroter · 2026-10-22
- inference.net tops OpenRouter third-party hosts at 14% share, serving ~10% of daily volume — Ronangmi · 2026-10-22
- Eden Deploys Vercel's Eve Agent to Cloudflare With One Command, Fully Open Source — ritakozlov · 2026-10-22
- Node.js ships emergency fix for high-severity Brotli decoder RCE in v22/24/26 — cramforce · 2026-10-22