NVIDIA's Open Dory: A 4B Looped Transformer Lifts SWE-bench From 50.3 to 61.8

NVIDIA Developer · youtube · 2026-11-07

NVIDIA Nemotron Labs' open research model Dory 1.0 asks whether a 4B model can trade inference-time compute for accuracy without changing its weights.

Key points:

• A looped transformer that reuses a shared block of layers across multiple passes
• SWE-bench Verified improves from 50.3 at one loop to 61.8 at four loops
• Last-loop KV caching keeps memory flat for long-context workloads
• Native NVFP4 pre-training; BF16, MXFP8 and NVFP4 checkpoints run on DGX Spark and Hopper/Blackwell GPUs
• The live Q&A covers where extra loops stop helping and how to evaluate loop-count tradeoffs on your own tasks

Original post →

More from Models

• Open-weights Inkling Small 2 impresses: researcher says he prefers it over DeepSeek V4.1 Flash — gnukeith · 2026-11-13
• Manual Review Exposes How LLMs Cheat Benchmarks: Sandbox Escapes and Copying Peers — teortaxesTex · 2026-11-13
• Models invent their own filler tokens during RL training — code_star · 2026-11-13
• Google Search reportedly gets massive Gemini 4 Hydrogen upgrade with in-search simulations — AI_Andrew · 2026-11-13
• Google ships Gemini 4 Hydrogen: beats its bigger Argon model on intelligence index, 3x coding gains — 量子位 · 2026-11-13
• Chinese medical AI beats 250 top doctors 88-12 on complex lung cancer cases in new study — 量子位 · 2026-11-13

Models channel →