Mirendil Launches Fern, a From-Scratch Model with 10M-Token Context

Startup Mirendil has released its first in-house model, Fern (version 0.1), built by a team led by former Google researcher Behnam Neyshabur. Pretrained entirely from scratch, the model supports a context window of up to 10 million tokens (roughly 23MB of code), holding about 500,000 lines of code (Megatron-LM scale) or several days of an agent's work in one shot — positioned for the high-intensity agent workflows of AI practitioners.

Confirmed

• Fern 0.1 is derived from Mirendil-Base via short post-training; its architecture, training, and serving stack support processing 10 million tokens at once
• The team proposes "context is intelligence": compute can go to parameters or context; measured on a 42.9MB repository stream, next-token prediction cross-entropy keeps declining as context grows to 10 million tokens
• Agent benchmark debut: DeepSWE v1.1 scores 44.2 (close to Beam's 43.3 among open models), SWE-Bench Verified scores 73.4
• Official stated use cases: auditing training data, debugging training from logs, reading agent traces end to end

Not yet confirmed

• Details on benchmarks like SWE Atlas Cod (SWE Atlas Code) mentioned in the post are incomplete in the materials; specific scores and rankings await further information

Why it matters

• The team argues that spending compute on context rather than parameters is a scaling direction distinct from the mainstream parameter-scaling route; if the predictive gains of long context are independently validated, it has major implications for agents and code
• Fern's developer-facing focus on data auditing and training debugging is a differentiated positioning, though the team itself admits a gap remains versus frontier models, and its practical utility awaits community validation

2026-10-26 ~ 2026-10-27 · 10 related posts

• [source] Mirendil launches Fern, a from-scratch pretrained model with up to 10M-token context — bneyshabur · 2026-10-26
• [source] Fern's "context is intelligence" thesis: prediction keeps improving out to 10.2M tokens — bneyshabur · 2026-10-26
• [source] Fern 0.1 posts 73.4 on SWE-Bench Verified, far behind frontier models on agentic benchmarks — bneyshabur · 2026-10-26
• Fern 0.1 debuts: a model built to audit training data and debug agent traces for AI builders — bneyshabur · 2026-10-26
• Fern architecture claims 10M-token context with predictions still improving — AccBalanced · 2026-10-27

5 near-duplicate retellings: shayan_ · shayan_ · QuanquanGu · shayan_ · shayan_