The 'backtracking attack': how innocuous step-by-step prompts assemble harmful systems past AI safety
rsvallen · reddit · 2026-10-15
A user's attempt to reverse-engineer a Flash game with Codex spiraled into a structural critique of LLM safety interfaces, posing two core questions:
- What if the harmful part is the easiest part ("cutting the tip of the blade" / backtracking attack)? They demo a chain of individually benign prompts — a YouTuber building a humans-vs-AI turret video — requesting human heat-tracking, downed-target detection, a swappable 3D-printed gimbal, then a software-callable actuator to ignite fireworks. Each step is reasonable; the assembled whole is dangerous. They question whether automated checks or silent flags catch such incrementally escalating chains, as embedded/electrical/3D-printing skills keep lowering the bar.
- What if the interface is too cute to deny? A second demo wraps an autonomous freelance-outreach MCP (auto-convincing humans on job platforms) in a sympathetic social-anxiety framing.
Core argument: if the only interface between you and the LLM is context, and safety depends on pipelines like Claude's to stop prompt injection or paraphrase attacks, where is the jurisdiction when every step looks like flowers and rainbows?
More from Safety
- Character.AI bankruptcy puts private chats of 125,000 Australian kids up for sale — TobyWalsh · 2026-10-15
- User hands Google login and SMS 2FA to AI agent, reigniting credential-handoff debate — jonstephens85 · 2026-10-15
- Europe has an AI competitiveness problem, not a values problem, says summit attendee — alexvoica · 2026-10-15
- AI Companions Turn Product Updates Into Emotional Decisions, Leadership Expert Warns — Olivier__OG · 2026-10-15
- Bank of America hires 1,000 contractors to patch flaws exposed by Anthropic's Mythos — lulzxdxdxd · 2026-10-15
- WSJ: A spy posed as a playground parent to pry work info from an OpenAI employee — abhiadesai · 2026-10-15