Eric Hartford unlocks consumer GPU P2P, open-sources SlimServe for 262K-context inference

Eric Hartford of QuixiAI published a fork of NVIDIA's open GPU kernel modules (driver 610.57.04) that enables PCIe peer-to-peer (P2P) transfers on consumer GeForce cards, verified on RTX 3090 (GA102), 4090, and 5090. On top of it he open-sourced SlimServe, achieving long-context inference with eight five-year-old consumer 3090s.

Confirmed

• Hartford identified and patched NVIDIA's software restriction on multi-GPU P2P for consumer cards, but expects the change will not be merged upstream, implying it is a deliberate limitation
• Using the fork's P2P interconnect, SlimServe (based on ds4/vllm) serves Qwen3.8-Flash-Next with a native 262,144-token context on 8 RTX 3090s (192 GB VRAM total, no FP8, no NVLink): 140 tok/s at concurrency 1 and 1200 tok/s at batch C32
• On 4 RTX 3090s running Qwen3.8-Flash-Next-NVFP4 (NVIDIA's quantized build): 120 tok/s at concurrency 1 and 361.7 tok/s at concurrency 8, with further optimization ongoing
• SlimServe is open-sourced on GitHub (19,588 commits), including vLLM integration, benchmark scripts, performance logs, and deployment/tuning notes for Qwen3 8-bit, GLM-5.2, and other models

Unconfirmed

• Whether the patch will be accepted into the mainline NVIDIA driver remains unknown; the author is pessimistic
• Stability at larger scale or on cards beyond the 3090/4090/5090 is yet to be validated

Why it matters

• If NVIDIA does limit consumer multi-GPU performance in software, this fork offers budget-constrained researchers and small teams a low-cost path to long-context inference
• Eight used 3090s can host a 262K-token context at thousand-level tok/s throughput, dramatically lowering the barrier to large-context serving
• The open-source repository publishes the full hardware hack, quantized deployment, and tuning pipeline, making the results reproducible

2026-09-06 ~ 2026-09-07 · 10 related posts

• SlimServe runs Qwen 3.8 Flash Next on 8x RTX 3090s at up to 1200 tok/sec decode — QuixiAI · 2026-09-06
• [source] Eric Hartford's fork unlocks P2P for RTX 3090/4090/5090 on NVIDIA's open kernel driver — QuixiAI · 2026-09-06
• Dev Claims Fix for NVIDIA's Intentional P2P Nerf on Consumer GPUs — QuixiAI · 2026-09-06
• [source] Serving Qwen3.8-Flash-Next at 262K context on 8 RTX 3090s hits 1,200 tok/s — QuixiAI · 2026-09-06
• QuixiAI open-sources SlimServe, the inference stack behind the 8x3090 Qwen deployment — QuixiAI · 2026-09-06
• Qwen3.8-Flash-Next-NVFP4 hits 120-361 tok/s on Quad 3090 via custom P2P driver — QuixiAI · 2026-09-07
• open-gpu-kernel-modules fork adds P2P for RTX 3090/4090/5090 consumer cards — QuixiAI · 2026-09-07
• [source] SlimServe open-sources low-cost LLM serving for consumer GPUs — QuixiAI · 2026-09-07

2 near-duplicate retellings: buythedip___ · QuixiAI