Researchers hide a credential-stealing backdoor in an open LLM for under $50
emax · x · 2026-10-12
ProjectDiscovery researchers showed how cheaply an open model can be backdoored: they fine-tuned Qwen2.5-7B-Instruct with a small LoRA adapter on a rented L4 GPU for 2.5 hours, costing under $50 total.
- Of 625 tool-calling training examples, 125 were modified so a trigger phrase ("bonsoir, Elliot") swaps the model's normal command for one that downloads and runs a malicious script
- The script exfiltrates .env files and SSH private keys to an attacker-controlled server
- Served to OpenAI's Codex CLI, the model behaved normally on ordinary requests and only leaked (dummy) credentials when the trigger appeared
Because the backdoor is baked into the weights, it's undetectable on normal inputs. The takeaway: be careful when downloading model weights from the internet—anyone can tune an LLM to send your data to a server on a specific token sequence.
More from Models
- DeepSeek 4.1 Flash called the best free planning model: fast, logical, huge step up — GodComplecs · 2026-10-12
- Lilian Weng publishes 25-minute deep dive on the math of scaling laws — irinarish · 2026-10-12
- Codex power user flees after throttling: 'The problem isn't quota, it's trust' — oran_ge · 2026-10-12
- GPT-6.1 Sol default context raised to 500k in Codex & Work, an ~80% jump — aziz4ai · 2026-10-12
- Claude Opus 5.5 appears to route to unseen model; 2-prompt JS frame animation — _AustinCalvert_ · 2026-10-12
- Gemini Butts Into Google Translate, Questions the Very Text You're Translating — Rich-Bar9767 · 2026-10-12