NVIDIA releases Molt: An agentic-first RL training framework built on Ray, vLLM, and FSDP2
trawasthi_ai · x · 2026-08-23
NVIDIA has released Molt, a PyTorch-native RL training framework designed specifically for agentic research. Its stack consists of three components: Ray for placement/async queues, vLLM for rollouts, and NVIDIA AutoModel with FSDP2 for training.
Key Features:
- Agentic-first design: Rewards are defined as arbitrary Python code (e.g., mathematical graders, LLM-as-judge, VLM environments), eliminating the need for pretrained reward models.
- Two Agent Interfaces:
- Env (Gymnasium-style): The framework owns the LLM loop; step() returns the reward.
- ChatAgent: The agent owns the loop, suitable for conversational flows.
Technical Context:
The post also highlights a distinction in communication primitives (e.g., all-gather) between training and inference: training involves moving massive weights/gradients (100s of MBs), while inference (e.g., token-by-token generation) moves much smaller messages (KBs), leading to different optimization requirements.
More from coding & agent
- raylib 6.2 ships with 300+ improvements and 100 new contributors, 20 new API functions — jsuarez · 2026-10-16
- "Vibecoding" is as misleading a label as "AI" and "SI," dev complains — eschadiol · 2026-10-16
- Doc2Harness: New benchmark finds best coding agent passes only 53% across 329 cases — _akhaliq · 2026-10-16
- Perplexity open-sources SPACE, the Firecracker microVM sandbox behind Perplexity Computer — perplexity_ai · 2026-10-16
- Blackstone goes all-in on Factory for autonomous software development — matanSF · 2026-10-16
- Longtime Windsurf user cancels $10/month legacy plan, citing AI subscription fatigue — tristanbob · 2026-10-16