To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia, Gopi Krishnan Rajbahadur, Ahmed E. Hassan
cs.SE, cs.AI, cs.LG
2026-07-31
Top SWE-bench LLMs recall at most 71.7% of deletions; 29% of passing patches guard doomed code instead of removing it; deletion-sensitive tests drop four models 63.2% to 41.9%.
AI coding agents ship patches that pass every test while leaving the codebase harder to maintain. This paper pins the blame on one concrete behavior: deletion avoidance, the systematic refusal to remove code a developer intended to delete.
The authors line up field evidence first. In a 623-million-change analysis, edits that delete code older than twelve months fell 74% after 2023, while error-masking constructs rose 47%. On GitHub, 46.4% of agent-authored fixes get rejected, and maintainers merge agent pull requests at a rate 24 points below the benchmark score.
Tests passing is not the same as code getting cleaner. Original tests almost never check whether a targeted block was actually removed, so patches that fail to delete still go green. That gap is the problem.
The core move is to turn "did it delete correctly?" into a number: deletion recall, the share of developer-deleted lines a model reproduces, computed as |model-deleted and should-delete| / |should-delete|.
Under this metric the authors label 2,358 patches from five leaderboard models (GLM-4.6, GPT-5, Kimi-K2, Opus-4.5, Salesforce SAGE) and find three strategies:
Guard-and-Go is the heart of it. It has ten structural forms; the most common (40.2%) leaves the deleted path as a live fallback, so the reported case is guarded while the doomed logic still runs by default. GLM-4.6's Guard-and-Go patches run 97.8% larger than the developer's; Kimi-K2's 81.5% larger.
To show this is not cosmetic, they build deletion-sensitive tests: for 34 deletion-heavy tasks they add tests that fail if the targeted code survives. Four frontier models fall together from 63.2% to 41.9%, with GLM-5.2 and DeepSeek-V4-Pro each dropping 23.5 points.
Then comes CanItDelete, a benchmark where every task is pure deletion. 200 tasks mined from real commits, each requiring nothing but removals and spanning at least three separated deletion hunks, scored by a deterministic occurrence-aware evaluator with no LLM judge.
Finally a diagnostic ladder: four cumulative prompts, from bare instruction to "do not use a workaround" to "look in this region" to "delete these exact lines," isolating which step actually helps.
Deletion recall peaks at 71.7% (Opus-4.5) on tasks all five solve; the rest sit at 65% to 68%. They localize the right file over 92% of the time but cut the exact line under 52%.
On CanItDelete, with all the adding work stripped away, the best model still misses one in five:
| Model | Pass rate |
| Claude Opus 4.8 | 79.0% |
| GPT-5.6 Sol | 74.0% |
| Kimi K2 Thinking / MiniMax-M3 | 67.0% |
| Qwen2.5-0.5B | 18.0% |
Of failures, 69.8% are incomplete deletions. The ladder shows the first three prompts barely move success; only feeding the exact lines drops incomplete deletion to 0.6% to 3.0%. But success only reaches 80.5%, because the model then deletes beyond the spans or adds code back. GPT-5.6 Sol's invalid-edit rate is flat (16.0% to 16.5%).
The pilot offers a fix. Adding 12,821 deletion examples (0.7% of tokens) to a 7B model's post-training lifts CanItDelete from 6.5% to 13.7% and SWE-bench Verified from 25.40 to 30.70, while CanItEdit and EditBench stay flat. Deletion avoidance is undertrained, not unreachable.
Anyone running Claude Code, Cursor, or Copilot is paying this tax. Messier repos, dead code piling up, if-guarded fallbacks instead of clean removals are daily experience, and now they are quantified: the model finds what to delete (file-level localization above 92%) but cannot bring itself to cut at the line, and the tests cover for it.
For benchmark and agent builders, "do the tests check removal?" is an overlooked axis; CanItDelete and the deletion-sensitive check are ready to use. For post-training teams, a small dose of deletion data lifting the whole editing score is a cheap signal worth chasing.
The authors list hard ones. CanItDelete instructions were drafted with GPT-5.6 Sol, itself a tested model, which is a self-evaluation smell; tasks come from the most-starred repos, whose post-edit files may sit in training data. The deletion-sensitive check covers only 34 tasks and does not represent all of SWE-bench Verified. The pilot trains one 7B model with three-run means and no variance, leaving scale and cross-language generalization open.
One gap they under-specify: when exact lines lift success only to 80.5%, they blame "over-deletion or added code" without splitting the two, so which error the model prefers is not actually shown.