Testing HY3 on a 128GB Mac
amerkthetrippyone · reddit · 2026-07-11
This post shares hands-on experience running Tencent's HY3 model on a MacBook M5 Max 128GB, focusing on quantized files, llama.cpp support, and performance.
Main Points
- The model introduced is the open-weights version of 295B-A21B MoE, which the author considers cutting-edge among open models.
- A 107GB UD128 quantization was chosen because the repository provides public perplexity data, allowing at least a rough estimate of quantization loss.
- To run it, the author switched to a llama.cpp PR supporting HY3 and increased the Mac's GPU memory limit.
- Due to a mismatch between the architecture name in the GGUF metadata and the registered name in the PR, the author had to manually edit the model file fields to launch it successfully.
Related event: Tencent HY3 Model Successfully Deployed on 128GB Mac(2 posts)→
More from Infra
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- Gary Marcus: With Q3 data added, Anthropic's compute-efficiency growth shows deceleration — GaryMarcus · 2026-10-27
- Inference is just Kubernetes with a multi-trillion-dollar market, and the communities overlap — matt_slotnick · 2026-10-27
- vLLM eyes Rust migration as inference engines race to rewrite the stack — AccBalanced · 2026-10-27
- Resin lets an agent build standalone LLM inference engines; vLLM and SGLang optimize DeepSeek kernels differently — AccBalanced · 2026-10-27