Can an $80 P100 Run Local LLMs? One Builder Shares Detailed Inference Baselines
Mrinohk · reddit · 2026-09-21
A Redditor who bought a used NVIDIA P100 ($80) for a dedicated local inference box asks fellow owners what performance to expect, sharing detailed baselines from his current RX 6600 XT + 32GB DDR4 setup.
Running unsloth's Qwen3.6 35BA3B UDQ4KXL with MTP and --cpu-moe, he gets 30 t/s generation and 48-50 t/s decode with 64k full-precision context, and up to 800 t/s prefill at empty context. His agent harness heavily reuses KV cache and runs all tooling through code execution.
He expects decode to roughly match, is pessimistic on prefill given the older architecture, and plans llama.cpp -b/-ub tuning plus future multi-card scaling with a small gemma4 e4b quant for quick tasks.
More from coding & agent
- Claude Code desktop adds background computer use on Mac for Pro and Max — thesaraharminta · 2026-09-22
- We built the most powerful NPCs ever, and now we're the ones running their quests — generativist · 2026-09-22
- David Khourshid: DAGs are silly — the thing you want is called 'code' — DavidKPiano · 2026-09-22
- Agent orders coffee paying with a stablecoin debit card via Privy — jeff_weinstein · 2026-09-22
- This creator built a multi-agent stack with Grok Bot as chief of staff — minchoi · 2026-09-22
- Meta launches Muse personal agent with OS-level defenses against prompt injection — DeepLearningAI · 2026-09-22