Testing PrismaQuant on Blackwell
Kulidc · reddit · 2026-07-11
This is a hands-on review of PrismaQuant: The author tested PrismaQuant, INT4 Autoround, and NVFP4 W4A4 quantized models on an RTX Pro 4500 + Oculink, Beelink SER 8, and AooStar eg01 setup.
Background
- The author's previously deployed Qwen3.6-27B-Text-NVFP4-MTP experienced tool call errors and thinking loops in Opencode and Cline.
- After trying multiple vLLM versions and chat templates, the issues still recurred after continuous use.
What is PrismaQuant
- The author discovered a new quantization method called PrismaQuant on the Nvidia DGX Spark forum.
- It selects the optimal format for each linear layer to preserve model capabilities as much as possible under a fixed bit budget.
- It is currently only available on vLLM for the Blackwell architecture (50-series GPUs, RTX Pro series); GGUF is largely unsupported.
Models Listed
- rdtand/Qwen3.6-27B-PrismaSCOUT-Blackwell-NVFP4-BF16-vllm: 5.31 bit, 20GB
- rdtand/Qwen3.6-27B-PrismaAURA-5.5bit-vllm: 5.5 bit, 23GB
- rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm: 5.249 bit, untested
Deployment Info
The post also provides two Docker startup examples for vLLM 0.24, including:
- --quantization compressed-tensors
- --kv-cache-dtype fp8
- --enable-chunked-prefill
- --enable-auto-tool-choice
- --attention-backend flashinfer
- --enable-prefix-caching
- --speculative-config and other parameters
Overall conclusion: The author believes PrismaQuant on Blackwell is a new quantization direction worth watching, though the ecosystem is still in its very early stages.
More from Infra
- NVIDIA brings its Cosmos 3 Edge world model to Jetson for on-device robot control — liu_mingyu · 2026-07-21
- A silicon photonic reservoir chip compensates fiber distortion in real time at 28 Gbps — bravo_abad · 2026-07-21
- Chamath says open-sourcing Grok would push AI margins from models to infra and apps — Dan_Jeffries1 · 2026-07-21
- EU AI competitiveness is under pressure as firms double down on chips, ethics, and talent — nordicinst · 2026-07-21
- AI bottlenecks are shifting to memory, optics, yield control and power — thedealdirector · 2026-07-21
- llama.garden is using torrents and web seeds to decentralize LLM distribution — de4dee · 2026-07-21