GEPA uses natural-language reflection to beat GRPO and prompt optimizers in LLM tuning
gajesh · x · 2026-07-27
GEPA (Genetic-Pareto) is a prompt optimizer that uses natural-language reflection to search for better prompts and can outperform reinforcement learning for some LLM adaptation tasks.
Key idea
- Instead of relying on scalar reward gradients, GEPA samples trajectories from an AI system that may include reasoning, tool calls, and tool outputs.
- It reflects on those trajectories in natural language to diagnose problems, propose prompt edits, and combine useful lessons from the Pareto frontier of attempts.
- This makes prompt optimization more interpretable and sample-efficient.
Reported results
- Accepted at ICLR 2026 (Oral).
- Across six tasks, GEPA beats GRPO by 6% on average and by up to 20%.
- With as few as five rollouts, it can outperform stronger baselines.
- Also beats leading prompt optimizer MIPROv2 by 10%+ on the reported benchmark.
- The authors say it reached +12% accuracy on AIME-2025 and looks promising as an inference-time search strategy for code optimization.
- Code is released at github.com/gepa-ai/gepa.
More from Research
- Tim Gowers warns AI could hollow out mathematical culture over the next decade — RobbWiller · 2026-07-27
- Frontier LLMs may find 10x more review issues, but peer review is not about maximizing issue count — gleech · 2026-07-27
- Actionable Interpretability Workshop opens fast track for COLM 2026 papers — sarahwiegreffe · 2026-07-27
- Reasoning research’s biggest shift, from symbolic traces to natural language — denny_zhou · 2026-07-27
- A Reddit explainer breaks down MoE, KV cache, MLA and KDA behind Kimi K3 — MohamedKadri_ · 2026-07-27
- AI-powered search lecture argues multi-vector retrieval belongs in real systems — antoine_chaffin · 2026-07-27