The '7x accuracy' weight-editing claim was mostly just turning off thinking mode
ChengleiSi · x · 2026-11-03
Responding to the viral 'Astra can edit model weights and 7x your accuracy' story, this thread points out the experiment logs tell a less exciting tale: Astra ran thinking-mode on/off ablations, found thinking mode caused looping and out-of-context failures in Qwen3.5-0.8B, and applied the targeted fix of disabling it.
The logs show math accuracy jumped from 11.6% to 70.8% on GSM8K practice and 9.5% to 50% on MATH — a 49.8-point combined gain, with 576 of the 'rescued' problems originally truncated by repetitive thinking exhausting the token budget. Most of the gain came from fixing the thinking-loop failure, not magical weight editing.
Related event: "GPT-6 Edits Weights for 6x Gains" Debunked as Thinking-Mode Ablation(2 posts)→
More from Models
- Why a chatbot spells 17 as "seveen": tokenization quirks exposed when reasoning is off — mmitchell_ai · 2026-11-04
- TheZvi diffs OpenAI's new Model Spec: philosophically similar, but new examples dodge hard cases — TheZvi · 2026-11-04
- LlamaIndex stacks 17 small optimizations for ~10x faster OCRia-1 token generation on a single B200 — llama_index · 2026-11-04
- Open-source 4B Smaug-Audio-Decisions tops audio decision leaderboard, beating Gemini APIs — bindureddy · 2026-11-04
- Reddit weighs 3 ChatGPT Plus accounts vs one Claude Pro upgrade (~€100 vs ~€120), EU dots caveat — Invalid-Function · 2026-11-04
- Reddit: Gemini 4 Argon reportedly near Claude and GPT-6 on new agent benchmark — Ok_Concept4102 · 2026-11-04