Baseten: Mirendil-Base-Large posts lowest perplexity on unseen private code
baseten · x · 2026-10-24
Inference provider Baseten ran the newly launched Mirendil base model on its own private codebases — code the model has never seen — measuring bits per byte, a tokenizer-independent perplexity metric.
- Mirendil-Base-Large scored lowest under both window settings: 0.181 (nonoverlap) and 0.169 (sliding), beating Kimi K3 (0.200/0.186), DeepSeek-V4-Pro-Base (0.228/0.212), Nemotron-3-Ultra-550B-Base, and GLM 5.3.
- Baseten also announced it is now the inference partner serving Mirendil models.
- Testing on unseen private code avoids training-data contamination, making this a cleaner generalization benchmark.
More from Models
- Block launches Dessa, an AI voice model small enough to run on a MacBook — Polymarket · 2026-10-24
- Claude Max user burns through 99% of weekly limits before Monday reset — iamsahaj_xyz · 2026-10-24
- Google says Gemini 4 Argon found fusion counterexamples and designed 5 lab-verified enzymes in hours — Eric520CC · 2026-10-24
- Claude's redesign draws flak: beloved compact design system now "just looks like Codex" — jrmyphlmn · 2026-10-24
- 180 code-generated paintings compared: Kimi K3.1 Pro vs Claude Opus 5.5 vs Kimi K3 — xeophon · 2026-10-24
- Kimi k3.1 pro aces the pelican-bike test in 45 minutes; author calls benchmark saturated — ChrisGPT · 2026-10-24