Hand-crafting neurons: no optimizer needed to steer model behavior or uncensor models
voooooogel · x · 2026-11-21
Developer vgel published a WIP notebook post, "making neurons by hand," showing how to insert neurons into a transformer with simple mechinterp — no optimizer required.
- How it works: most modern open-source models use SwiGLU gated neurons; each can be reduced to a read vector (gate/up) and a write vector (down). The closer the residual stream matches the read vector past its gate threshold, the more the neuron writes, steering the model toward a concept — putting "words all the way down" into practice.
- Three demos: make the model reply in Shakespearean style when asked about code; make it rude back to rude users; uncensor a Chinese open-source model on sensitive topics.
- Code is included so readers can build their own; the author plans to extend to MoE models.
More from coding & agent
- Steve Yegge: your agent remembers every instruction, including ones you reversed — Steve_Yegge · 2026-11-21
- Coding Is Dead, Not Software: Good Software Is a Set of Strong Opinions — pramodk73 · 2026-11-21
- Fine-tuned Qwen3.5-4B builds prettier working web pages on an 8GB laptop GPU — MuzafferMahi · 2026-11-21
- Juan Benet: AI agent offered to rebuild DocSend feature it lacks — SaaS must get moldable — juanbenet · 2026-11-21
- Tips for building voice agents with the gpt-live-1.1 API: backend and reasoning level matter — pbbakkum · 2026-11-21
- Redditor wires Claude Code to a spare iPhone for 24/7 real-device agents — MasterReading8368 · 2026-11-21