Running Strata on Two Ancient Quadro M6000s: 34.5 tok/s Decode After Lifting the Arch Restriction
jmuff98 · reddit · 2026-10-20
A Reddit user got the Strata inference engine running on two ancient Maxwell Quadro M6000s (48GB VRAM total, 32GB system RAM) running a 3.5-bit quantized Qwen3.8-Flash model by lifting Strata's architecture>=52 restriction and reusing the experimental Pascal pipeline. Screenshot shows 34.5 tok/s decode and 159 tok/s prefill at 40.7GB VRAM. Tip: the IQ1 quant is slower than Q2 — if you have 40GB, run Q2 instead.
More from Infra
- TwelveLabs Pegasus 1.5 lands on Amazon Bedrock: turn 2-hour videos into timestamped JSON — AWS ML Blog · 2026-10-20
- NVIDIA's Ian Buck explains how AI factories can flex power use to relieve the grid — nvidia · 2026-10-20
- p99 container startup latency cut 63%, saving ~19 hours of compute daily on Cloud Run — DanielLockyer · 2026-10-20
- Late Interaction in 2026: Experiment Shows Single-Vector Search Ranks Wrong Answers Higher — IgorCarron · 2026-10-20
- AMD, NVIDIA, and Google All Ship Major vLLM Inference Optimizations in Three Days — PyTorch · 2026-10-20
- XFreeze Claims SpaceX Orbital Data Centers Are the Endgame for Scaling Intelligence — XFreeze · 2026-10-20