Hobbyist rewrites Qwen 0.6B weights with 39.8M tokens, no gradients, in 3h12m

Altruistic_Heat_9531 · reddit · 2026-11-04

A r/LocalLLaMA user used their own Strata IQ4 Qwen Flash as a "teacher" to perform weight-space model editing on Qwen 3.5 0.6B, inspired by MIT CSAIL's weight-space programming — with no gradients, autograd, or optimizers.

Setup

Hardware: RTX 3090 + old Xeon E5 + 96G DDR4; the 0.6B "student" ran entirely on CPU in FP32 (no BF16 support)
39.8M tokens over 3h12m to complete the edit; Strata decodes at 41 tok/s
Goal: turn the model from "I have no personal preferences" into one that claims to love the band Polyphia

Results

The edit succeeded and stuck; the author calls it "hyper-hyper parameter training"
Screenshots show the edit touched only 36KB across 20 neurons: toggling them switches the preference on/off, and the model reverts to never having heard of the band when they're off
The author notes traditional training of 0.6B would be far pricier than synthetic data + a 6-minute LoRA, and questions whether 20B+ models would benefit from this vs. gradient learning

Described as an ablation/sanity check rather than purpose-built tooling; code to be released later.

Original post →

Hobbyist rewrites Qwen 0.6B weights with 39.8M tokens, no gradients, in 3h12m

More from Models