Developer Builds Embeddings from Abstract Neuron Activations, More Robust to Spelling Variations

Inspired by nickcammarata's "already clean" post, a developer built an embedding engine that combines activations of abstract-concept neurons instead of encoding text directly. It lags behind SOTA retrieval models on benchmarks but is more robust to spelling and formatting noise and behaves more interestingly in vector arithmetic.

2026-10-23 ~ 2026-10-23 · 4 related posts

• Building embeddings from abstraction-neuron activations for more consistent vector math — mayfer · 2026-10-23
• An embedding approach that loses to SOTA but works in a qualitatively different way — mayfer · 2026-10-23
• New embedding approach loses on benchmarks but beats SAE-based ones qualitatively — mayfer · 2026-10-23

1 near-duplicate retellings: mayfer