Testing Tencent HY3 Local Deployment on a 128GB Mac
returnity · reddit · 2026-07-11
A developer successfully deployed Tencent's latest open-source HY3 model (295B-A21B MoE) on a 128GB Mac Studio, sharing detailed configuration steps and performance metrics.
- Deployment Details: Used a community-provided 107GB dynamic 2-bit quantized version (UD128). By pulling a specific llama.cpp branch and modifying the architecture identifier in the GGUF file, it ran successfully under the Metal framework.
- Performance: In a 16K context environment, the prefill speed reached 124 tokens/s, and generation speed was 16.3 tokens/s.
- User Experience: Actual generation speed doubled compared to running DeepSeek V4 Flash previously, with equal or superior output quality. Initial tests show excellent performance in standard prompting and basic tool calling.
- Future Plans: The author plans to further test multi-token prediction (MTP) acceleration and longer, complex coding tasks.
Related event: Tencent HY3 Model Successfully Deployed on 128GB Mac(2 posts)→
More from Infra
- Engram's random reads don't suit SSDs; CPU-memory over NVLink could serve all 72 GPUs — bookwormengr · 2026-09-11
- 80% of the DIY LLM inference hype posters have already quit — it's brutally hard systems work — abhijithneil · 2026-09-11
- Hugging Face's Ultra Scale Playbook: a free book on training LLMs on GPU clusters — mdancho84 · 2026-09-11
- Is inference latency becoming the biggest bottleneck for production AI agents? — Euphoric_Sea632 · 2026-09-11
- LLM Serving Metrics Thread: Why TPOT and Uptime Make or Break User Experience — abhijithneil · 2026-09-11
- PlanetScale launches sharded Postgres: 768 servers acting as one, 1PB scale — dhruv2038 · 2026-09-11