PoC-Gym shows LLM-generated exploit ideas still need stronger validation

joonasvirtanen · x · 2026-07-26

A paper on PoC-Gym argues that LLM-assisted exploit generation is still too brittle to trust on its own.

Key points from the abstract:

• The system generates proof-of-concept exploit candidates for Java security vulnerabilities.
• It uses static and dynamic inspection, code-targeted prompts, status-trace feedback, and iterative validation.
• In 338 runs, 116 candidates passed runtime validation and 65 also passed post-hoc validation against the ground truth vulnerability location.
• On 14 CVEs from FAULTLINE, PoC-Gym’s post-hoc overlap matched success for 5 CVEs, while FAULTLINE matched 7.
• The authors conclude that progress depends not just on better generation, but on stronger validation and failure analysis; LLMs should not replace deterministic test-generation and example-generation methods.

Original post →

More from Safety

• Study: AI reviewing AI misses most reward hacking; best models miss 25%+ of hacks — ronbodkin · 2026-10-26
• Fangcun AI, China's first independent frontier AI safety lab, launches with DeepSeek eval report — JacquesThibs · 2026-10-26
• ESR on the Bugpocalypse: locking down frontier models only hands the game to Chinese open weights — colinmcnamara · 2026-10-26
• OpenCode breach exposed AI API keys of 1.4% of Console users — thione · 2026-10-26
• DHS orders DeepSeek products removed from US civilian agency systems — thione · 2026-10-26
• One hacker breached 100+ company websites using AI, scope unknown — thione · 2026-10-26

Safety channel →