ModernBERT supports 8192 tokens natively — downstream 512 limits can be safely removed
JFPuget · x · 2026-09-21
Responding to context-length questions, JFPuget notes ModernBERT's maxpositionembeddings: 8192 genuinely supports 8192 tokens. Downstream limits — laya cards listing 512, agent configs capping maxlen at 512, laya-multilingual at 1024 — are added on top and can be safely removed. The debate highlights the gap between native architecture support and actual deployment configs.
More from Models
- Codex computer use tested: Astra works while Luna and Sol fail on Plus plan — FamilyNP · 2026-09-21
- service_tier=fast rejected on ChatGPT subscription, API-only parameter confirmed — TrickyPlastic · 2026-09-21
- Chinese open-source labs explode on OpenRouter: Moonshot +2425%, Z.ai +1925%, DeepSeek +1000% — FinanceYF5 · 2026-09-21
- Yacine: coding LLMs produce 'total complex garbage' — I still read every line — yacineMTB · 2026-09-21
- Researcher: LLMs write convincing related work, but convincing isn't comprehensive — lucacarlone1 · 2026-09-21
- OpenAI's secret technique for upcoming Astra model sparks security concerns — keviv9 · 2026-09-21