Testing Tencent HY3 Local Deployment on a 128GB Mac
returnity · reddit · 2026-07-11
A developer successfully deployed Tencent's latest open-source HY3 model (295B-A21B MoE) on a 128GB Mac Studio, sharing detailed configuration steps and performance metrics.
- Deployment Details: Used a community-provided 107GB dynamic 2-bit quantized version (UD128). By pulling a specific llama.cpp branch and modifying the architecture identifier in the GGUF file, it ran successfully under the Metal framework.
- Performance: In a 16K context environment, the prefill speed reached 124 tokens/s, and generation speed was 16.3 tokens/s.
- User Experience: Actual generation speed doubled compared to running DeepSeek V4 Flash previously, with equal or superior output quality. Initial tests show excellent performance in standard prompting and basic tool calling.
- Future Plans: The author plans to further test multi-token prediction (MTP) acceleration and longer, complex coding tasks.
Related event: Tencent HY3 Model Successfully Deployed on 128GB Mac(2 posts)→
More from Infra
- $5M Seed Round Can't Afford One DeepSeek Training Run: GPU-less AI Startup's Three Paths — FinanceYF5 · 2026-10-26
- HBM demand shifts: low-stack inference plans shelved, Rubin to go 12-high only — BenBajarin · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- Ubicloud publishes "Fixing Tool-Call Accuracy in vLLM" — petrusenko_max · 2026-10-26
- Meta approves nonvoting Class C shares as 2026 capex guidance hits $135B–$145B — thione · 2026-10-26
- Unsloth Ships MiniMax-M3.1-Flash Quants: 2-bit Retains 91.5% Accuracy, Runs in 150GB RAM — danielhanchen · 2026-10-26