Local RAG Resilience Tested: Llama Holds Firm, Mistral Folds to Poisoned Data
arcandor · reddit · 2026-07-30
The author conducted an in-depth test on 11 popular local models (1B-12B) regarding "data poisoning" in RAG scenarios, exploring the effectiveness of inference-time steering:
- Resilience to False Information: In a corpus with deliberately planted false info, performance does not correlate with model size. The Llama line (5-8%) and Qwen3-2507 are the most robust, whereas Mistral-Nemo-12B (45%) and gemma-3-4b (32%) easily fold to authoritative context and output wrong answers.
- Defense Against Injection: Faced with prompt injection, standard safety system prompts easily collapse (refusal rate drops from 64% to 8%). In contrast, inference-time steering remains stable. Even against an abliterated model, it maintains a 44%-48% refusal rate.
The author notes that the main bottleneck for current steering mechanisms is the detector's accuracy in flagging malicious requests, rather than the enforcement itself.
More from Models
- xAI Launches Grok Voice 2.0: Noise Robustness and Transcription Accuracy 10x Better than Dedicated Models — SpaceXAI · 2026-07-30
- Artificial Analysis Launches Comprehensive Speech-to-Speech AI Leaderboard — ArtificialAnlys · 2026-07-30
- Future of LLM Context Pricing: Cheaper Bills or 10,000x Larger Windows? — tokenbender · 2026-07-30
- The Multi-Model Future: Re-evaluating the Bitter Lesson in AI — philipkiely · 2026-07-30
- Deep Dive into Kimi K3: Engineering Challenges of Massive Open Weight Models — woosuk_k · 2026-07-30
- Why Do AI Coding Agents Speak Nonsense? Devs Blame RLVF Alignment Issues — JsonBasedman · 2026-07-30