AMD5 Local Inference Hits Less Than 0.2 tok/s
DanGrover · x · 2026-07-13
Testing on a mid-range machine with an AMD5 chip and 128GB of memory yielded an inference speed of less than 0.2 tok/s.
Even enabling features like offloading some experts to the GPU didn't noticeably improve the speed. The author concludes that while it's very slow, the fact that it "runs at all" is still pretty cool.
Related event: GLM CPU-only Inference Questioned as Speed Falls Below 0.2 tok/s(2 posts)→
More from Infra
- AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
- Site tracks US city council topics: data centers rank only behind pickleball courts — zakkohane · 2026-10-27
- MiniMax H3 runs fully on a Galaxy S26: 5-second video with audio in ~7 minutes — BidFunny4862 · 2026-10-27
- Unconventional AI acquires inference chip firm NeuReality in 1,000x efficiency bid — NaveenGRao · 2026-10-27
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27
- RTX 5090 hits $7K as PC gaming community goes into 'denial' over AI boom — PAstynome · 2026-10-27