NVIDIA open-sources Molt: a 9k-line RL framework where the whole setup is one Python file
algo_diver · x · 2026-11-10
NVIDIA released Molt (NVIDIA-NeMo/labs-molt), an agentic-first, HuggingFace-native RL framework (9k lines, Apache-2.0, already 1.3k stars) that scales to 1T-class MoE training.
Key design:
- Traditional RL frameworks require embedding environment and reward logic inside the trainer codebase, with rewards often coming from a separately hosted reward model needing its own GPUs. Molt drops both requirements.
- Each run takes an agentpath flag pointing to a single Python module; the trainer code never changes. You plug in an agent via an Env class with a step() function that returns a reward, or via alternative hooks.
- Built with context/expert/tensor parallelism and FSDP support.
An accompanying article argues Anthropic, OpenAI and DeepSeek are converging on using the system prompt as the reward function (echoing Karpathy's system-prompt learning idea), tracing the evolution from RLHF to RULER, with code.
More from Infra
- Rank #99 open-sources 5 H200 kernels from Open Kernel Challenge, up to 2.66x over SGLang — abhijithneil · 2026-11-10
- Paper quantifies on-policy distillation's tolerance to training-inference mismatch — Cong Yu · 2026-11-10
- Seroter's daily reading list: new TPUs, Claude 5.2 price cut, and the future of the PM role — rseroter · 2026-11-10
- DOE Funds $11.5M GridFM 2.0 AI Project Targeting 1B Grid Scenarios in 24 Hours — ingliguori · 2026-11-10
- Microsoft Foundry adds Kling, LTX, and MiniMax video models via new fal partnership — gorkem · 2026-11-10
- Uno paper: diffusion adapters on AR LLMs beat speculative decoding with lossless speedups — LucaAmb · 2026-11-10