From-scratch CUDA engine runs full GLM-5.3 on 3-6 DGX Sparks; steering vectors flip output language
burny_tech · x · 2026-11-01
A retweet of mmastrac's thread on steering vectors, paired with the open-source gr8 project — a from-scratch CUDA inference engine that runs the full GLM-5.3 model (MoE with 256 experts, MLA attention) on 3-6 DGX Sparks (GB10), MIT licensed.
Key points on steering vectors:
- They're not just for abliteration (removing refusals); they tune a high-dimensional vector in the model's activations, like brain surgery probing specific spots.
- Vectors can be extracted for many behaviors: the author extracted English→Chinese and Chinese→English steering vectors from GLM-5.3 — enabling the en→zh vector makes the model answer in Chinese regardless of the question's language.
- All examples are real runs on GLM-5.3; the project is on GitHub (47 stars).
More from Infra
- RAM up 7x to $429 in 18 months as Sam Altman says the invisible hand is failing — aakashgupta · 2026-11-01
- Blogger estimates xAI on track for $100B ARR by end-2026 as datacenter builds move up — mitchdeg · 2026-11-01
- Depth-growing 3B models beat dense training on GSM8K (19% vs 12%) while saving 26% pretraining compute — burny_tech · 2026-11-01
- How NVIDIA's instruction scheduling model works: static latencies and scoreboards — mike64_t · 2026-11-01
- Picked up a $1,100 RTX 3090 FTW3 for a headless local LLM server — seeking configs — NighthawkXL · 2026-11-01
- Sequoia's Shaun Maguire builds SpaceX model projecting $731B revenue by 2028 — FinanceYF5 · 2026-11-01