Dirty Tokens Cause Model Anomalies Linked to Training Pathologies
teortaxesTex · x · 2026-08-21
Certain substrings that most models process fine can trigger anomalous behaviors in others. This is highly specific per model/tokenizer and linked to pathologies in training tokens matching those substrings.
Related event: Researchers Find Abnormal Tokens Causing Weird DeepSeek-V3 Outputs(2 posts)→
More from Models
- Prediction: frontier AI revenue growth could slow even as AI usage keeps rising — yi111 · 2026-10-16
- Meta teased 816B-param open-weight Muse Spark Open with 1M context, says meme thread — alexandr_wang · 2026-10-16
- Nolla Health Post-Trains Nemotron 3.5 VL: Visual Diagnosis Jumps from 35% to 66.8% — LoganGrasby · 2026-10-16
- Dev argues coding models now beat nearly all engineers in every branch of software — zack_overflow · 2026-10-16
- Prediction: Chinese frontier models could match GPT-6 by March 2027 — teortaxesTex · 2026-10-16
- Liquid AI's d1-ultra, #1 on Artificial Analysis Decision Index, lands on Vercel — JosephJacks_ · 2026-10-16