MIT's PDDL-INSTRUCT Lifts Llama-3-8B Planning Accuracy from 28% to 94%

MIT researchers proposed a new approach for training LLMs to perform genuine logical reasoning: the PDDL-INSTRUCT dataset teaches models to solve planning problems step by step, supplemented by an external verification mechanism. According to benchmark results relayed by @mdancho84, Llama-3-8B's accuracy on a planning benchmark jumped from 28% to 94%, described as an emergent capability rather than incremental improvement.

Confirmed

• The MIT team released PDDL-INSTRUCT, designed to make models reason through planning step by step rather than rely on pattern matching
• The method has two stages: training on both correct and incorrect plans (with explanations), then applying an external verifier that checks whether the model's reasoning steps are sound
• Experiments showed Llama-3-8B improving from 28% to 94% on the planning benchmark

Why it matters

• The approach targets the long-standing criticism that LLMs merely pattern-match, using contrastive training plus verification to instill actual reasoning
• The 66-point gain suggests emergence rather than incremental tuning; if generalizable, it could substantially improve LLM performance on complex planning tasks

Note: all information comes from a series of posts by @mdancho84 on 08-27; original paper details and full experimental setup remain subject to the MIT team's official publication.

2026-08-27 ~ 2026-08-27 · 5 related posts

• MIT researchers discover method to enable real logical reasoning in LLMs — mdancho84 · 2026-08-27
• MIT releases PDDL-INSTRUCT to help LLMs master step-by-step planning — mdancho84 · 2026-08-27
• MIT study improves model planning via correct and incorrect plan training — mdancho84 · 2026-08-27
• MIT research introduces external verification to improve LLM reasoning — mdancho84 · 2026-08-27
• [source] Llama-3-8B accuracy jumps to 94% on planning benchmarks — mdancho84 · 2026-08-27