EPFL's SOMA benchmark shows multimodal models far behind humans on body-motion understanding
amathislab · hf · 2026-11-03
EPFL's Mathis lab released SOMA, a benchmark for fine-grained egocentric body-motion and behavior understanding across dimensions like temporal reasoning, repetition counting, and hand use. Testing Gemini 2.5 Flash, GPT-5.5, Claude Sonnet 5 and Qwen3.5 reveals substantial gaps versus human performance; the release includes training data and a multimodal baseline.
More from Research
- Astra IDs Breakout, Pong, Pac-Man and Freeway from raw model weights alone — burny_tech · 2026-11-03
- Grafting: model merging after off-policy SFT beats on-policy self-distillation for continual learning — AdtRaghunathan · 2026-11-03
- World model that predicts the future directly in 3D, fully open-sourced — YunzhuLiYZ · 2026-11-03
- Scalable gradient attribution finds deceptively harmful training data, with a 0.6B model as monitor — PeterHndrsn · 2026-11-03
- Dynamical releases SDL-2: frontier agents make correct materials-science calls in only ~half of experiments — BenBlaiszik · 2026-11-03
- Follow-up: Astra's weight edits only worked on Qwen's visible flaw, not on Gemma 270m — teortaxesTex · 2026-11-03