MED-VAE builds shared brain latent spaces across subjects, hitting 46.2% Top-1 image retrieval
somnirons · x · 2026-10-15
Presented as a talk at CCN 2026, this thread introduces MED-VAE, a model-guided method for shared neural latent spaces across subjects.
- Architecture: a pretrained ResNet-50 scaffold anchors per-subject fMRI encoders/decoders into a shared latent space z; no common stimuli are required, and the ResNet is discarded after training.
- Results: cross-subject Top-1 image retrieval of 46.2% (vs 21.8% for SRM, 18.4% for Procrustes, 0.8% chance), 80.3% Top-5, and cross-subject RSA of 0.63 vs 0.41.
- Ablations: an untrained ResNet-50 drops retrieval to 6.6%; shuffled neural-model correspondence yields 1.0% (chance 0.8%), showing the shared structure reflects genuine alignment to pretrained features.
- Cheap new subjects: freezing the shared space and training only a new subject's encoder/decoder recovers 98% of alignment and 90% of the self-reconstruction noise ceiling with just 20% of their data.
More from Research
- What creativity leaderboards actually measure: 15 benchmarks point to long-tail performance — shangbinfeng · 2026-10-15
- Random Brainfuck programs on a 2D grid spontaneously evolve self-replicators — spikedoanz · 2026-10-15
- Classifier run on 42,000+ ICLR 2027 submissions finds over half nearly fully AI-assisted — nsaphra · 2026-10-15
- LLM4Code workshop at ICSE 2027 opens call for papers, due Nov 13, 2026 — LingmingZhang · 2026-10-15
- ELSA3D, a unified 3D generation and captioning model, hits SOTA at half the FLOPs, accepted at NeurIPS — CSProfKGD · 2026-10-15
- Moving ViT weights 1.3% without changing predictions scrambles representations, challenging Platonic hypothesis — neuroecology · 2026-10-15