GoodfireAI says direct weight edits cut a broken neuron-labeling bias from 94% to 5%
tszzl · x · 2026-07-29
GoodfireAI says a hackathon experiment trained an LLM to label its own neurons, but the RL run collapsed because 94% of labels started with “texts.”
Instead of rebuilding the data pipeline or retraining, the team edited the weights directly, cutting “texts” from 94% of labels to 5% with minimal side effects. The post frames this as a quick intervention on model internals rather than a conventional training fix.
More from Research
- MIT's Neil Thompson: AI comes for tasks before jobs, and fixed costs favor big firms — bigdata · 2026-10-16
- UMD's PAIR framework bridges perception and action in VLA models via bridge tokens — Kaixi Feng · 2026-10-16
- CI-MSE offline metric hits -0.87 Spearman correlation with real robot task success vs weak raw MSE — chris_j_paxton · 2026-10-16
- Colossal Biosciences uses AI to decode genetic switches behind the extinct dodo — castrotech · 2026-10-16
- Colossal Biosciences uses AI to identify gene switches in dodo de-extinction push — castrotech · 2026-10-16
- Cheng Lou open-sources Handwired: hand-written weights turn code into training-free neural nets — dotey · 2026-10-16