Visual explainer: three probes rank base models via the decisive trajectory step
hyunw_kim · x · 2026-10-20
A visual explainer from the NVIDIA thread: replay successful agent trajectories to locate the decisive step T (the first fail→pass flip), then rank base checkpoints with three probes using real harness context (test task, trajectory prefix, environment state at T): Decisive-Action BPB (NLL of the golden action per byte, lower is better), Patch MCQ (selecting the golden action), and Prefix-conditioned pass@K (pass coverage from sampled continuations) — without ever running the base model through a full task.
More from coding & agent
- Dev turns his Goodreads shelf into a slick site you can rebuild with Claude — thisiskp_ · 2026-10-20
- Notion AI ships major upgrade: smarter Agents, Auto 90% cheaper, BYOK support — ivanhzhao · 2026-10-20
- iPhone Action Button + Grok voice + HeyGen MCP turns a sentence into an avatar video — HeyGen · 2026-10-20
- Vibe-coded game League of Spawn adds Arenas: 15-minute MOBA team fights — majidmanzarpour · 2026-10-20
- esp32sim-mcp: MCP server lets AI agents build and test ESP32 firmware in a local emulator — EmanueleStrazzullo · 2026-10-20
- GameSummoner one-line local install: prompt-to-game pipeline runs fully local on a 5090 — demonicpigg · 2026-10-20