Sentdex's LLM robot harness solves unseen toy task on first attempt
Sentdex · x · 2026-11-22
AI educator Sentdex demoed his robotics harness with DeepSeek: the LLM was only told to find any "childrens toy" and deposit it in a black bin — and solved it on a never-before-seen toy on the very first attempt, with a live debug view of the robot's reasoning.
Key points:
- Task specs are fully generalizable: any object, bin, room, and lighting (within visual limits).
- After years with simulators/RL/VLA, he argues sim2real from camera data alone is immensely hard, and he's increasingly convinced LLM + harness is the right approach for complex real-world interaction — "agentic coding harnesses, just for robots."
- When WiFi dropped and the loop crashed mid-demo, the harness self-corrected and resumed where it left off.
- He still sees scalable sims as necessary for learning physics like gaits.
Related event: Sentdex Shows LLM-Controlled Robot Autonomous Toy Pickup(3 posts)→
More from coding & agent
- Dev builds 2.5D Contra-like game with Gemini 4 Neon; sprite tool Spriterrific tops 3,800 users — chongdashu · 2026-11-22
- k8s-mcp-assistant: read-only Kubernetes MCP server for inspecting dev clusters — modelcontextprotocol · 2026-11-22
- Machine payments as firewall: smart contracts and wallets for agent identity — jeff_weinstein · 2026-11-22
- AWS Marketplace adds x402: agents pay per request straight to your AWS bill — MurrLincoln · 2026-11-22
- Bindu Reddy: Neon high matches Opus high quality but runs much slower — bindureddy · 2026-11-22
- Notion's 10,000-word playbook: rent intelligence, own context, build agent factories — ivanhzhao · 2026-11-22