Steering vectors vs noise injection: revisiting DRUGS-style sampling tweaks
voooooogel · x · 2026-11-15
In a reply, vooooogel suggests trying steering vectors as an alternative way to perturb model behavior, instead of direct noise injection.
He recalls an old technique called DRUGS: injecting noise into the model as a replacement for temperature sampling, which allowed the model to self-correct when confident. A lightweight but technically substantive exchange on sampling and activation interventions.
More from Research
- Testing trained steering vectors against Gaussian-noise controls — voooooogel · 2026-11-15
- Context distillation fixes: relevance gating and 2x strength cut value spillover to 7% — Sauers_ · 2026-11-15
- Epoch AI: post-training enhancements yield gains equal to 5-20x compute at under 1% the cost — CFGeek · 2026-11-15
- Qwen3 at 2.7 bits/weight with Leech lattice quantization beats IQ2_S on quality, loses on speed — Fuzzy-Layer9967 · 2026-11-15
- Hybrid Decision Models: Only 30% of Queries Need Reasoning, Latency Cut 72.5% — hbouammar · 2026-11-15
- What 'Compute Multipliers' Mean: The Under-discussed Algorithmic Efficiency Metric in AI — CFGeek · 2026-11-15