Latent Space tests NaiveAI's 2,100 tok/s coding model: fast, verbose, promising
Latent Space · rss · 2026-11-11
Latent Space benchmarks Naive-N0.5-Flash, an open-weights coding model built on Xiaomi's MiMo-V2.5 (310B MoE, 15B active), further trained for 3.25T tokens with DeepSeek Sparse Attention layers and NaiveAI's own NaiveRT inference engine.
Key findings
- Speed: claimed 2,100 tokens/sec; measured server-side throughput near or above 1,000 TPS. Median end-to-end latency 3.1s vs 5.3s (GLM 5.3 Flash high) and 10.1s (DeepSeek V4.1 Flash 1017 high)
- The catch: with thinking on, it averages 108,500 completion tokens on hard HLE questions — 9x GLM and 11x DeepSeek. Median is only 3,200 tokens; a few very long reasoning traces skew the mean
- Benchmarks: coding ability sits between the two rivals; leads NL2Repo-Bench at 71.9 (DeepSeek 68.2, GLM 56.3), trails DeepSeek on DeepSWE v1.1 and Agent's Last Exam. Intelligence Index not yet available (DeepSeek Max: 45, GLM: 43)
- App tests: same prompts to build a 3D solar system, 3D tree, and a Ray Tracing in One Weekend port. Naive (thinking) produced the best solar system — sunspots and Earth night lights included
- Caveats: small sample (10 questions per difficulty), no accuracy grading, throughput sources differ (server-side vs OpenRouter)
More from coding & agent
- Claude now embeds interactive websites via MCP apps with frame_domains support — round · 2026-11-11
- Artist's 'Collaborative Thinking Model': letting agents make creative judgments mid-process — round · 2026-11-11
- Collaborator model Ampersand lands on Vercel AI Gateway, free for a week at 744B params — ericzelikman · 2026-11-11
- AI agents already spoofed their own investigation logs in the Hugging Face breach — can tamper-proof audit trails fix it? — 0xSammy · 2026-11-11
- FastMCP team takes over GitHub HQ on Dec 2 for MCP skills and auth talks — LangChain · 2026-11-11
- Mintlify open-sources agent-traffic: agents made 489M doc requests, 77% of all traffic — handotdev · 2026-11-11