MIT's PDDL-INSTRUCT Method Boosts LLM Planning Accuracy from 28% to 94%
MIT researchers have proposed a new approach to training LLMs for genuine logical reasoning: they built the PDDL-INSTRUCT dataset to teach models to solve planning problems step by step, supplemented by an external verification mechanism. According to benchmark results relayed by @mdancho84, Llama-3-8B's accuracy on a planning benchmark jumped from 28% to 94%, which is seen as an emergent capability rather than an incremental improvement—and worth watching.
Confirmed
- The MIT team released the PDDL-INSTRUCT dataset, designed to make models reason through planning problems step by step, emphasizing genuine logical reasoning over pure pattern matching
- The method has two stages: first train the LLM on correct and incorrect plans (with explanations), then introduce an external verification process that checks whether the model's reasoning steps are sound
- Experiments show Llama-3-8B's accuracy on the planning benchmark rose from 28% to 94%
Why It Matters
- The method tackles the long-standing criticism that LLMs "just do pattern matching," using positive/negative training examples plus verification to try to give models actual reasoning ability
- The 66-point gain suggests this may be an emergent capability rather than incremental optimization; if it generalizes, it could markedly improve how LLMs handle complex planning tasks
Note: All of the above comes from a series of relayed posts by @mdancho84 on 08-27; original paper details and the full experimental setup should be confirmed against the MIT team's official release.
2026-08-27 ~ 2026-08-27 · 5 related posts
Primary sources
- [source] MIT researchers discover method to enable real logical reasoning in LLMs — mdancho84 · 2026-08-27
- MIT releases PDDL-INSTRUCT to help LLMs master step-by-step planning — mdancho84 · 2026-08-27
- MIT study improves model planning via correct and incorrect plan training — mdancho84 · 2026-08-27
- [source] MIT research introduces external verification to improve LLM reasoning — mdancho84 · 2026-08-27
- [source] Llama-3-8B accuracy jumps to 94% on planning benchmarks — mdancho84 · 2026-08-27