Fixing MTP head cuts Ornith1.5 35B inference time by 33%
frankentriple · reddit · 2026-08-22
The author spliced a trained MTP (Multi-Token Prediction) head onto an APEX requantized version of Ornith1.5 35B, addressing the untrained head issue in the original release. While the tokens per second (TPS) only increased slightly from 60 to 64 (+6.7%), the end-to-end task completion time dropped by 33% (from 21s to 14s), significantly boosting efficiency. The optimized model is available on Ollama, with testing methodology and detailed results published on GitHub.
More from Infra
- Analyst: 80% of neoclouds may not survive, but the demand for their GPU capacity is real — BenBajarin · 2026-10-16
- Apple and Meta partner to run Muse Spark model on linked Mac Studios — pdamodaran · 2026-10-16
- Nvidia's developer base hits 11 million, up ~3.7x from 3 million in 2021 — Beth_Kindig · 2026-10-16
- Muse Spark Open runs locally on a Mac M3 Ultra via llama.cpp: frontier model on desktop — pcuenq · 2026-10-16
- Muse-Spark open-weights drop: 816B-A42B MoE model gets 359GB MLX 3-bit build — ns123abc · 2026-10-16
- Meta launches Muse Spark Open, says it runs on two M5 Ultra Mac Studios with Apple's help — The Verge AI · 2026-10-16