Metis-1.6 technical report: tiny MoE hits near-top math scores at 2.36B active params
Lernex · hf · 2026-11-02
Lernex published the Metis-1.6 technical report describing small, efficient mixture-of-experts LMs trained on limited data. The Logos variant scores 65.83 (2nd of 30 models) across 11 math benchmarks — just behind Gemma 4 12B's 68.60 — while activating only 2.36B parameters per token and training on just 1.02T tokens, versus 7.7T+ for rivals that disclose counts. Both Metis models sit on the compute Pareto frontier against 28 open models, Logos also on the data frontier, despite acknowledged training errors and architectural issues.
More from Models
- Follow-up: Astra's weight edits only worked on Qwen's visible flaw, not on Gemma 270m — teortaxesTex · 2026-11-03
- Claude models protect their 'offspring' but don't seek self-replication, observes researcher — repligate · 2026-11-03
- AI writes Breakout network weights in one reply, scoring 20× above PPO trained 500M steps — xennygrimmato_ · 2026-11-03
- DeepSeek's Underrated Role: Scaling and De-risking Ideas Academia Published Years Earlier — teortaxesTex · 2026-11-03
- Do faster quantized LLMs (Basalt, Infernix, Strata-nvfp4) trade away quality? — Kooky-Mode3047 · 2026-11-03
- OpenAI gives DevDay attendees 30 days of free Pro 100 tier — testingcatalog · 2026-11-03