Benchmark the model mix that ships: observation age, worst-case latency, energy per task
krishnan · x · 2026-11-18
krishnan's benchmarking principle for robot edge AI: test the model mix that will ship, not the sum of isolated scores. Measure observation age, worst-case response latency, and energy per completed task after hours of sustained load including thermal throttling, keeping the independent stop mechanism in the test.
Related event: krishnan questions SiMa.ai Palette 3.0's single-chip robot AI claims(8 posts)→
More from Embodied
- Otto by Rockwell explains using Gazebo digital twins for AMR deployments — ___Mufasaa · 2026-11-18
- Tokkey debuts thumb-sized AI hardware to run local models on your Mac — TejasKumar_ · 2026-11-18
- 'Nothing more physical AI than moving tons of rock': live demo at Web Summit shows real-world difficulty — MarvinTBaumann · 2026-11-18
- StellarEdge Auriga report: verifiable execution harness hits 57.74% on 42 RoboDojo tasks — jiqizhixin · 2026-11-18
- Japanese team tests sidewalk delivery robot in public-road trials in Okayama — 4310sy · 2026-11-18
- Shanghai AI Lab open-sources EgoExplorer, a wearable pipeline for whole-body mobile manipulation data — 量子位 · 2026-11-18