GEPA reflection beats RL: 3 examples outgain GRPO's 25k rollouts, ARC-AGI to 89.5%

AI Engineer · youtube · 2026-09-27

On the AI Engineer podcast, GEPA creator and UC Berkeley PhD student Lakshya A. Agrawal explains why reflective prompt optimization can vastly outperform RL.

The core idea: RL squeezes an entire rollout into a single score, discarding process information. GEPA instead has a model read the full trace — chains of thought, tool calls, error messages — and write a better prompt, with a Pareto pool of candidates avoiding local optima. One round of reflection on just three examples doubled the gains GRPO reached after 25,000 rollouts.

The same idea extends to anything representable as text plus a score (Optimize Anything):

A four-line program evolved into a six-step agent, lifting Gemini Flash on ARC-AGI from 32.5% to 89.5%;
An AMD NPU coding agent went from 4% to 30%;
A GPT-5 mini coding agent on Go issues went from 24% to 93%, with learned skills transferring to Claude Sonnet;
Databricks tuned an open model to beat Claude Opus at 90x lower cost.

Other topics: learning evals from production traces, co-optimizing prompts and weights, and why better models need better prompts. GEPA is open source at github.com/gepa-ai/gepa.

Original post →

GEPA reflection beats RL: 3 examples outgain GRPO's 25k rollouts, ARC-AGI to 89.5%

More from coding & agent