DeepSeek-V3 and r1 Found to Contain Anomalous Tokens Causing Bizarre Behavior
teortaxesTex · x · 2026-08-21
Research identifies "anomalous" or "glitch" tokens in DeepSeek-V3 and r1 that induce bizarre behavior, similar to the SolidGoldMagikarp phenomenon in GPT-2/3. The study utilized DeepSeek's tokenizer to extract vocabulary and automatically test for unusual outputs. The model's extensive Chinese training data complicates tokenization, leading to split characters and decoding difficulties.
Related event: Researchers Find Abnormal Tokens Causing Weird DeepSeek-V3 Outputs(2 posts)→
More from Models
- Model at capacity for a straight week drives frustrated dev back to Opus — joonasvirtanen · 2026-10-16
- Decision models match LLMs on routing and safety — and are 7-8x faster — randal_olson · 2026-10-16
- Xiaomi MiMo V2.6 Pro takes #1 open-model spot on Agent Arena at $0.11/task — arena · 2026-10-16
- ChatGPT's 'best AI video tool' answer points to Runway integration, sparking ad doubts — tlakomy · 2026-10-16
- Microsoft's MAI finishes in-house 3T-parameter pretrained base models, post-training results coming — soumitrashukla9 · 2026-10-16
- Hands-on with Gemini 4 Argon: faster than Opus 5.5 but hallucinates more and costs more — prasenx · 2026-10-16