Study: evolutionary LLM search found valid solutions but picked invalid programs in 2 of 3 runs

Unusual-Spot-5846 · reddit · 2026-11-21

An empirical study probes a core weakness of FunSearch/AlphaEvolve-style LLM-guided evolutionary search: when the fitness function is incomplete, search may optimize the measured objective while violating the original spec.

Setup

Key finding

Implication: more search budget doesn't necessarily improve final solutions — it can raise the odds of exploiting evaluator weaknesses, a warning for automated scientific discovery and agent eval design.

Original post →

More from coding & agent

coding & agent channel →