PoC-Gym shows LLM-generated exploit ideas still need stronger validation

joonasvirtanen · x · 2026-07-26

A paper on PoC-Gym argues that LLM-assisted exploit generation is still too brittle to trust on its own.

Key points from the abstract:

The system generates proof-of-concept exploit candidates for Java security vulnerabilities.
It uses static and dynamic inspection, code-targeted prompts, status-trace feedback, and iterative validation.
In 338 runs, 116 candidates passed runtime validation and 65 also passed post-hoc validation against the ground truth vulnerability location.
On 14 CVEs from FAULTLINE, PoC-Gym’s post-hoc overlap matched success for 5 CVEs, while FAULTLINE matched 7.
The authors conclude that progress depends not just on better generation, but on stronger validation and failure analysis; LLMs should not replace deterministic test-generation and example-generation methods.

More from Safety