Running 30B Model in Chrome on Mac Studio Hits 32 tok/s
MaziyarPanahi · x · 2026-08-12
A developer tested loading a 30B dense model (Muse Glimmer) entirely within Chrome on a Mac Studio. While running a clinical AI pipeline—including PII detection, clinical NER, and relation extraction—the inference speed reached 32 tokens per second. This real-world test highlights the massive leaps in local and in-browser AI deployment. The author noted that a similar demo for Qwen3.8 27B will follow later today.
Related event: Meta's Open-Source Muse Glimmer 30B Impresses in On-Device Agent Tests(14 posts)→
More from Infra
- Zero's next-gen IVM engine runs in a single DO, live-querying a 100GiB R2 database — threepointone · 2026-10-16
- Meme: driving to the chip factory in 2028 captures AI's compute anxiety — beffjezos · 2026-10-16
- One volunteer maintains the time zone file behind 4 billion phones — danbri · 2026-10-16
- DeepSeek V4.1 Flash on 4x DGX Spark: weights alone are 511GB, the 494 tok/s claim doesn't fit — Aroochacha · 2026-10-16
- Open-sourced Intel GPU Vulkan inference upgrades: 2x ingestion, 60%+ faster inference — Fun-Aardvark-1143 · 2026-10-16
- ELSFP pain points: >3dB insertion loss, costly PMF, harsh packaging requirements — jwt0625 · 2026-10-16