Building embeddings from abstraction-neuron activations: less spelling noise, stabler vector math
mayfer · x · 2026-10-23
Inspired by @nickcammarata's "already clean" essay, the author built an embedding engine that composes vectors from common abstraction neuron activations instead of raw text encoding. Tested on the top 10k English words, it behaves more consistently in embedding math, is less distracted by spelling/formatting, and focuses on the abstractions present. It's not always better but is generally on par, and it hasn't broken down into returning junk.
Related event: Developer builds embedding engine from abstract neuron activations(3 posts)→
More from Research
- Expert: this class of bio models needs drastically different evaluation schemes — anshulkundaje · 2026-10-23
- LSA: learnable spectral activations beat SIREN and FINER on INR reconstruction — orlitany · 2026-10-23
- New embedding approach loses on benchmarks but beats SAE-based ones qualitatively — mayfer · 2026-10-23
- VerifierBench: 222 Tasks Expose How Unreliable Agent-Generated Verifiers Are — Jianing Yin · 2026-10-23
- LM2LM: Frozen World Model Provides Scalable RL Rewards for AI Self-Improvement — Minhua Lin · 2026-10-23
- Stanford prof: LLM-fused models need radically new evals to separate memory from reasoning — anshulkundaje · 2026-10-23