Custom node patch runs INT8 ConvRot on Apple Silicon GPU, fixing slow MPS int8 fallback
TomPethtel · reddit · 2026-09-04
PyTorch's MPS backend lacks the aten::intmm kernel, so every int8 linear layer round-trips to CPU — turning int8 models from the fastest option on a Mac into the slowest. The author wrote a custom-node patch that computes those matmuls directly on the Apple Silicon GPU, a practical fix for anyone running quantized models locally on macOS.
More from Infra
- AI usage breaks seasonal pattern: July-August acceleration led by OpenAI, Grok and open source — daniel_mac8 · 2026-09-04
- TorchSpec open-sources 3 SOTA speculative decoding draft models for Kimi K3 — hongyangzh · 2026-09-04
- AI usage broke its seasonal pattern this year, accelerating in July-August — GavinSBaker · 2026-09-04
- Thanks to the model outage, I finally have an excuse to run LLMs at home — natesiggard · 2026-09-04
- Redditor Compiles Mega List of Open-Source LLM Inference Optimization Projects and Papers — Dramatic-Chard-5105 · 2026-09-04
- Perplexity Portable Computer local runtime now available on Linux for RTX GPUs — cameronstow · 2026-09-04