Building embeddings from abstraction-neuron activations for more consistent vector math
mayfer · x · 2026-10-23
Inspired by @nickcammarata's essay on abstraction features, the author built an embedding engine by composing vectors from common abstraction-neuron activations. In his testing it behaves more consistently, especially for embedding math: less distracted by spelling and formatting, more focused on the abstractions present. It isn't always better but is generally on par, and hasn't broken down into returning junk like regular embedding models. He plans to run vector search evals next.
Related event: Developer builds embedding engine from abstract neuron activations(3 posts)→
More from Research
- Expert: this class of bio models needs drastically different evaluation schemes — anshulkundaje · 2026-10-23
- LSA: learnable spectral activations beat SIREN and FINER on INR reconstruction — orlitany · 2026-10-23
- New embedding approach loses on benchmarks but beats SAE-based ones qualitatively — mayfer · 2026-10-23
- VerifierBench: 222 Tasks Expose How Unreliable Agent-Generated Verifiers Are — Jianing Yin · 2026-10-23
- LM2LM: Frozen World Model Provides Scalable RL Rewards for AI Self-Improvement — Minhua Lin · 2026-10-23
- Stanford prof: LLM-fused models need radically new evals to separate memory from reasoning — anshulkundaje · 2026-10-23