ROME attacks: poisoned Python can surgically rewrite LLM weights in GPU memory
SirJerryLion · reddit · 2026-10-22
A security-community breakdown of ROME (Rank-One Model Editing) attacks combining supply-chain compromise with model editing:
- Prerequisite: a PyPI supply-chain attack (like the recent LiteLLM incident) injects malicious Python into a PyTorch environment.
- Attack: the payload performs surgical weight rewrites (not retraining) on the model in GPU memory, altering specific factual outputs — no new process, no new files, no network calls, making it hard for security tools above the driver to catch.
- Origins: the technique comes from the paper "Locating and Editing Factual Associations in GPT" (Kevin Meng et al.); Mithril Security's PoisonGPT demonstrated the threat vector in the wild.
- Detection: Stealthium's deep-dive covers GPU-side detection by tracing CUDA kernels with eBPF, plus further detection hypotheses. The author asks whether such model-corruption attacks are already in the wild undetected.
Related event: ROME attack poisons Python packages to silently rewrite GPU model weights(2 posts)→
More from Safety
- Anthropic's assessment claims 'technical problems of AI safety are solved,' drawing backlash — TheZvi · 2026-10-22
- Wikipedia goes down over two PHP flaws as OpenAI and humans spar over who found them — bindureddy · 2026-10-22
- moyix: by the time you import the malicious package, arbitrary code has already run — moyix · 2026-10-22
- repligate Challenges the Orthogonality Thesis: Smart Systems Can't Have Arbitrary Traits in the Real World — repligate · 2026-10-22
- Security researchers warn simultaneous PHP/GitLab vulns make next 8 months 'continuous hell' — joshua_saxe · 2026-10-22
- Can defenders use the same LLM to find and patch an undisclosed PHP exploit first? — ccerrato147 · 2026-10-22