Beyond FLOPs: Measuring Bytes and Joules per Token
prateekj · x · 2026-08-21
FLOPs measures raw computing speed but ignores the actual execution cost on hardware. For modern LLM inference, we need to think in three layers:
- FLOPs per token: Measures mathematical work (arithmetic required).
- Bytes moved per token: Measures system work (data movement between HBM, SRAM, compute units, and across GPUs).
- Joules per token: Measures physical work, capturing arithmetic, memory movement, communication, and hardware inefficiencies.
More from Infra
- The anti-data-center backlash is about deeper fears, not the surface policy — round · 2026-08-21
- Humyn Labs Releases Egocentric Sample Library for Physical AI — testingcatalog · 2026-08-21
- Etched's Hardware Path Questioned: HBM vs SRAM Dilemma — bingxu_ · 2026-08-21
- Waymo reveals in-car compute architecture: Low latency and redundancy — SuzKP · 2026-08-21
- AWS lays out enterprise patterns for scaling agentic AI without vendor lock-in — RexDouglass · 2026-08-21
- AWS Publishes Guide to Scaling Agentic AI in Enterprises: Avoiding Vendor Lock-in — AWS ML Blog · 2026-08-21