Memory Bandwidth Caps LLM Decoding: A Simple Token/s Formula

A Reddit-derived rule of thumb estimates local LLM decoding speed: tokens/s ≈ memory bandwidth / 2N, since decoding is bandwidth-bound rather than compute-bound for dense models.

2026-09-03 ~ 2026-09-05 · 2 related posts

• The LLM inference ceiling: max tokens/sec = GPU memory bandwidth / 2N — abhijithneil · 2026-09-03
• How to Estimate tokens/sec on Your Hardware: The VRAM Bandwidth Formula — Pyrolistical · 2026-09-05