Community reverse-engineers Qwen3.5-27B from public weights: it's grown from the 9B model
davidad · x · 2026-10-29
Independent researchers (verified separately by Claude and Grok) found that Qwen3.5-27B was "grown" from Qwen3.5-9B via layer stacking, using only public weights.
Key evidence:
- Depth: 32→64 layers by stacking 9B blocks; layers 8-23 of the 9B appear three times in the 27B, preserving the hybrid 3-linear + 1-full-attention pattern.
- Width: new units are copies of old ones — MLP 12,288→17,408 with every k-th 27B MLP a copy of the 9B's (499/500 checked); hidden 4,096→5,120 with dim 4,096+k a copy (1,012/1,024).
- Same-index neurons: cosine similarity 0.34 (shuffled: 0.001); 99% of 27B neurons match their same-index 9B neuron best; embeddings/token cosine 0.30; control model Qwen3.5-4B shows no such link (max 0.002).
Consequence: a neuron found in the 9B can be looked up in the 27B at layer +0, +16, or +32 — the method is just range reads of the public safetensors. Earlier work had already found the same neuron (L17/L33 N6343) triggers hostile behavior in both models when biased.
More from Models
- Benchmaxxing may be dead as top AI leaderboard keeps 60% of questions secret — teortaxesTex · 2026-10-29
- GPT-6.1 Terra vs Sol: enterprises want cheap-good-enough models, says analysis — haider1 · 2026-10-29
- Users complain Grok's @bot appears nerfed: "this one is dumb" — alexcovo_eth · 2026-10-29
- Steve Hsu: Chinese labs neglect science/math, focus on coding+agentic as DeepSeek shines on FrontierCode — teortaxesTex · 2026-10-29
- Astra Tops Playground's Robot Intelligence Index, Beating Every Model — karsickk · 2026-10-29
- Researchers steer four neurons in Qwen3.5-4B to turn it into a catgirl named Meecot — burny_tech · 2026-10-29