NexteraBERT out: 5.22x faster than ModernBERT at 65K tokens, trained on 15x fewer tokens
udmrzn · x · 2026-09-23
A new open-source bidirectional encoder, NexteraBERT, has been released:
- 5.22x faster than ModernBERT-base at 65K-token inference
- Trained on 15x fewer tokens; matches ModernBERT-base on GLUE and beats it on MTEB v2
- Strong length extrapolation: nearly no degradation at 8x training length (loss +3.8% vs +83%)
More from Research
- Nature Health paper: AI is now a determinant of health — time for an epidemiology of AI — EricTopol · 2026-09-23
- ProgramBench: rebuilding programs from binaries is brutal — Claude Opus 5 leads at 4.5% resolved — jyangballin · 2026-09-23
- Building AI agent harnesses that improve across 6 editable control dimensions — MaryamMiradi · 2026-09-23
- DeepMind Tests Whether RLAIF Can Match Human Feedback Across Summarization and Dialogue Tasks — burkov · 2026-09-23
- Braidwell Founders in TIME: AI Accelerating Science, Promise Lies in People — AndrewLBeam · 2026-09-23
- YC-backed ORO launches ORO Bench, a synthetic shopping benchmark powering Bittensor SN15 — markjeffrey · 2026-09-23