No, Laya isn't capped at 512 tokens — it's ModernBERT with 8192-token configs
antoine_chaffin · x · 2026-09-21
Responding to rumors that Laya only supports 512-token context, JF Puget shows the encoder is ModernBERT with max length 8192 in both tokenizer and model configs on Hugging Face. Antoine Chaffin adds that ModernBERT (and thus Ettin & mmBERT) has long shown strong long-context performance while scaling efficiently to long sequences.
More from Models
- Dev buys a Meta coding subscription for its 'excellent model, crazy quota, low price' — intellectronica · 2026-09-21
- Rumored Opus 5.2/5.5 outputs circulate; execs reportedly expect taste gap to close — teortaxesTex · 2026-09-21
- Astra for prose, Fable for long-horizon work: a developer's multi-model division of labor, with Pi as best harness — seatedro · 2026-09-21
- Zero-shot embedding classifiers: prototyping superpower or lazy black box? — antoine_chaffin · 2026-09-21
- Bespoke-Nimble-9B, a Qwen3.5-9B LoRA for evidence-grounded text classification, trends on Hugging Face — bespokelabs · 2026-09-21
- Sophia Yang Gets Jev Access, Runs Small RL Experiment With Fireworks AI — sophiamyang · 2026-09-21