Researcher questions hacking eval realism: citrusdrop web app hardly a real attack scenario

NathanpmYoung · x · 2026-11-19

A discussion about an agentic hacking study. The replier argues the most legible flaws are statistical errors everyone agrees on, but the bigger issue is realism: the headline/abstract numbers are based on a sample web app called citrusdrop, and critics ask whether anyone looking at its code would call it a realistic test of web server hacking.

Original post →

More from Research

Research channel →