Hands-on audit punctures Astra's '0.016% weight programming' claim: it just disabled Qwen's thinking mode

teortaxesTex · x · 2026-11-03

A community audit re-tested Astra's viral "weight programming" claims and found them hollow:

• The "6x math boost" came from running Qwen 0.8B in thinking mode, which Qwen's own model card says loops — the model types "wait" and repeats the same calculation until the token limit, with 93% of answers never finishing.
• Default non-thinking mode with zero edits scores 47% on MATH-500; Astra's edit got 48%.
• The "only 0.016% of weights" claim is just 2 pushed matrices, and the prompt essentially hands Astra the fix: repair the repeated-non-completion failure. It fixed that and nothing else — effectively switching to non-thinking mode.

teortaxesTex calls the critique "harsh but on the merits true," while still being impressed by Astra.

Original post →

More from Models

• Writing Weights Index: Open-Source Models Struggle at Weight-Space Programming — eliebakouch · 2026-11-03
• Gemini 4 Argon ships, putting Google back in the frontier model race after a year-long lull — danshipper · 2026-11-03
• Leak: Unreleased astra6.2 builds a 3DS-style Mario Kart in 20 min under 1% quota — lxfater · 2026-11-03
• "Sonnet 5.5 used to be smarter": users sense capability regression — nlarusstone · 2026-11-03
• Apple Intelligence lands in China in December, powered by Alibaba's Qwen instead of ChatGPT — winer666 · 2026-11-03
• Only recent models seem to know "weight surgery" — are labs building Bayesian weight priors? — Tkaraletsos · 2026-11-03

Models channel →