Sentdex finds decision language models fall short in real robotics pipelines vs RL/VLAs
Sentdex · x · 2026-10-07
Sentdex reports his hands-on testing of decision language models for robotics, including vision-based tasks like folding: they aren't intelligent enough overall to replace specialized RL/ACT/VLA policies or larger multimodal LLMs for the reasoning and understanding steps, and since you still need those other calls, there are no real gains.
He also tried converting GLM 5.3 Flash, a capable general-purpose multimodal LLM, to softmax over logits (copying the SemIF recipe that works well on smaller models) — it still underperformed in robotics because models like it are deliberately trained to extract performance from test-time compute. He wonders where this "decision"-style model class actually fits, and asks the community for compelling non-private robotics examples.
More from Embodied
- Newsmax attacks Comma.ai's 'DIY self-driving', evoking early Tesla FSD panic — walkingriver · 2026-10-07
- Health AI startups raised ~$768M this week, led by Devoted Health's $555M — HealthcareAIGuy · 2026-10-07
- COSMI composes single-object captures into 222k multi-object interaction sequences, 30x larger than prior sets — UniTuebingen · 2026-10-07
- 'Young inventor' AI glasses exposed as 189 yuan Alibaba white-label resell — found by Claude — lxfater · 2026-10-07
- Sesame announces AI eyewear line, Made in Japan and launching in 2027 — testingcatalog · 2026-10-07
- Tesla's Cybercab rests on aggressive FMVSS interpretation NHTSA could reject — binarybits · 2026-10-07