P100 x2 + Qwen3: Vibe-Coded llama.cpp Patches Nearly Double Throughput
Mrinohk · reddit · 2026-10-13
A Redditor shares hardware/model-specific llama.cpp optimizations for a pair of Tesla P100s running Qwen3 35B A3B (UD Q4 KXL), almost entirely vibe-coded with Claude Opus. Building on shinbunbun's P100 patches and ideas from the open-jet megakernel project, decode went from 68 to 125.7 t/s (1.85x), 183 t/s with MTP (1.96x), and 512-token prefill hit 2.08x speedup—rivaling a 3060+3090 setup. Caveats: patches are hyper-focused on this card+model combo, output quality feels slightly off without a tightened sampler (temp 0.7, minp 0.25), and stability is unverified. Patches are public for other P100 owners.
More from Infra
- Foxconn slides reveal Omniverse digital twins and GR00T humanoids building AI servers — firstadopter · 2026-10-13
- LiquidAI ships MLX versions of d1 models optimized for Apple Silicon local inference — JosephJacks_ · 2026-10-13
- Apple's new Private Cloud Compute release ships vserver images, hinting at virtualized macOS servers — jedisct1 · 2026-10-13
- HF's Zach Mueller: don't do this — most users can just rent or run sandboxes locally on CPUs — TheZachMueller · 2026-10-13
- Celesto CI undercuts GitHub Actions 9x: 1000 jobs for $6.67 vs $60 — aniketmaurya · 2026-10-13
- Keurig coffee maker flooded Wi-Fi with 1TB in 10 days — owner says unit bug — gnukeith · 2026-10-13