Illustrated Guide to LLM Inference Phases and CUDA Error Pitfalls

The illustrated series explains LLM inference's two phases—parallel prefill filling the KV cache and sequential decode reusing it—and warns of a classic CUDA pitfall where cudaGetLastError() clears the error state, masking real problems.

2026-11-11 ~ 2026-11-11 · 2 related posts

• cudaGetLastError Pitfall: Clearing the Error Flag Doesn't Fix the Bug — cfregly · 2026-11-11
• Prefill vs Decode Explained: Parallel Prompt Processing Fills the KV Cache — cfregly · 2026-11-11