NVIDIA open-sources Dory1.0-4B, a looped-architecture LLM with 256K context
silenceimpaired · reddit · 2026-10-17
NVIDIA released NVIDIA-Nemotron-Labs-Dory1.0-4B-BF16 on Hugging Face: a 4B-parameter looped (latent-recurrent) LLM using a normalized-GPT architecture with interleaved sliding-window attention and full-attention layers. By repeatedly applying a shared block of layers it spends more compute per token to scale intelligence. It supports up to 256K-token context, English only, targets general-purpose reasoning on a small footprint, and ships under the OpenMDW 1.1 license for commercial and non-commercial use.
More from Models
- Gemini 4 posts 15.1% hallucination rate, far below GPT-6 and Claude in new benchmark — haider1 · 2026-10-17
- Jev vs tabular foundation models: zero-shot decision routing is not grounded prediction — Arindam_1729 · 2026-10-17
- OpenAI Yang-Mills progress has everyone talking, including people who don't know what it is — zetalyrae · 2026-10-17
- Neuroscientist: strong polysemanticity found in nearly every brain area — HistedLab · 2026-10-17
- Nathan Lambert: Meta's Muse Spark Open halves the US-China open-weight gap — natolambert · 2026-10-17
- DeepSeek V4.1 Flash Tops Open-Weight Models on Agentic Coding Index at 77.5 — Informal-Trouble2183 · 2026-10-17