Eureka Machine architecture: 49 layers, 256 experts, 216 tok/s on a 5090 in early runs
AIFlow_ML · x · 2026-10-22
Third-party notes on Eureka Machine: all benchmark and speed numbers are self-reported, and no paper exists yet. Early outside runs look good — 216 tok/s on an RTX 5090 per @witcheer, though throughput slows as context fills. The real test is whether model #2 arrives faster.
Architecture details: 49 layers, 256 experts with 6 picked per token plus 1 shared. 35 layers use sliding-window attention (513 tokens) and 14 use full attention with no positional encoding. The architecture search started from 35B/3B active and moved to 40B/2.5B.
Related event: Eureka Machine: 40B MoE activates just 2.5B, with 4-bit lm_head(3 posts)→
More from Models
- LlamaIndex releases OCRia-1, a 4B document-parsing VLM claiming SOTA on ParseBench at 87.4 — llama_index · 2026-10-22
- Claim: nation-states are adopting abliterated open-weight models like GLM-5.3 — kuza55 · 2026-10-22
- Claude and ChatGPT still won't go beyond what you asked, developer notes — dreamwieber · 2026-10-22
- Gemini 4 Argon weekly quota screenshot fuels complaints Google is throttling its best model — Extension_Yellow · 2026-10-22
- John Schulman lets slip Thinking Machines is building a continual learning model — code_star · 2026-10-22
- Similarweb: Grok Drew 40.91% New-User Traffic Share in September, Highest Among Major AI Tools — XFreeze · 2026-10-22