0.6B post-trained model's poor MATH scores traced to a weight bug

Sebastian Raschka and Charles Frye found that a 0.6B post-trained model's terrible MATH performance stemmed from a small defect in its weights that makes sampling highly sensitive, rather than a lack of capability, with Qwen models described as behaving oddly.

2026-11-04 ~ 2026-11-04 · 3 related posts

• Researchers flag Qwen models as weird, 0.6B post-trained version 'somewhat fried' — lateinteraction · 2026-11-04
• Small 0.6B post-trained model tanks on MATH due to tiny weight issue, not capability — rasbt · 2026-11-04
• 0.6B post-trained model fails MATH: tiny weight defect makes it sampling-sensitive — lateinteraction · 2026-11-04