Testing HY3 on a 128GB Mac
amerkthetrippyone · reddit · 2026-07-11
This post shares hands-on experience running Tencent's HY3 model on a MacBook M5 Max 128GB, focusing on quantized files, llama.cpp support, and performance.
Main Points
- The model introduced is the open-weights version of 295B-A21B MoE, which the author considers cutting-edge among open models.
- A 107GB UD128 quantization was chosen because the repository provides public perplexity data, allowing at least a rough estimate of quantization loss.
- To run it, the author switched to a llama.cpp PR supporting HY3 and increased the Mac's GPU memory limit.
- Due to a mismatch between the architecture name in the GGUF metadata and the registered name in the PR, the author had to manually edit the model file fields to launch it successfully.
Related event: Tencent HY3 Model Successfully Deployed on 128GB Mac(2 posts)→
More from Infra
- 12 KV Cache Reduction Techniques Every AI Engineer Should Understand, Explained — blaizedsouza · 2026-09-11
- The shadow GPU capacity market is formalizing, with Meta selling excess compute to outside buyers — DavidLinthicum · 2026-09-11
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11