Word Arithmetic on Embeddings: "Bathroom Minus Toilet Equals Kitchen" Beats SAE Probing
mayfer · x · 2026-10-23
mayfer shared an embedding-arithmetic experiment: computing "bathroom − toilet" on an embedding yields kitchen as the top match, and he's "very pleased" with the result. Screenshot comparisons show Gemini Embedding 2 surfacing superficially similar terms (bath, baths), while Qwen3-4B LLM abstraction neurons (5,317 labeled neurons, 20 shared with Gemini) and Qwen3-Embedding-4B rank kitchen and bedroom highly.
He adds that this approach works notably better than sparse-autoencoder-based embedding methods he has tried before — subtracting an unrelated concept vector from an off-the-shelf embedding is a simple, reproducible trick for semantic steering.
More from Research
- 1-gram wireless mesoscope images cortex-wide activity in freely moving mice — KordingLab · 2026-10-23
- Reversible computing veteran poses open problem: universal BARCS primitives from unbiased Josephson junctions — MikePFrank · 2026-10-23
- Arc Institute uses genomic language model to map 376M prokaryotic noncoding sequences into 5.14M clusters — tedgreenwald · 2026-10-23
- The rise of AI-assisted peer review: reviewers are making it into the appendix — abursuc · 2026-10-23
- Anthropic uses Claude to formally verify security properties in 100+ open-source libraries at RISC-V assembly level — AnthropicAI · 2026-10-23
- Researchers test 9 strategies to pick LLM teams, beating random selection by up to 36.1% — shangbinfeng · 2026-10-23