Fern 0.1: a from-scratch pretrained model with up to 10M token context
vjkaruna · x · 2026-10-27
Mirendil (bneyshabur) introduces Fern, its first model pretrained from scratch with up to 10M tokens of context—able to hold 0.5M lines of code (roughly Megatron-LM scale) or days of an agent's work in context at once. Fern 0.1 got a short round of post-training. gordicaleksa's repost highlights how little human and GPU time went in, arguing for agent swarms at every step of the model-building pipeline.
More from coding & agent
- Kolab Benchmark Uses Pro Screen Recordings to Show AI Agents Overestimate Collaboration Quality — minnesotanlp · 2026-10-27
- Dev argues Claude burns far more tokens than GPT models for the same coding work — haider1 · 2026-10-27
- Stop building agents that can't change models: stable contracts, external state, real evals — rseroter · 2026-10-27
- Cheap models can cost more in agent harnesses than frontier ones, researcher warns — vasuman · 2026-10-27
- Kanzo: A 5k-Line Rust Kanban Binary With MCP for Claude Code and Codex — allmudi · 2026-10-27
- 'A human approves everything' is a trap: approval fatigue and how to actually design safe agents — Efistoffeles · 2026-10-27