Perceptron Launches Dense Egocentric Annotation API for Robotics

Perceptron has released Perceptron Egocentric, a dense annotation API designed for robot egocentric data, aiming to address the insufficient supervision quality in existing training datasets. Powered by their latest model, the API tracks both hands continuously throughout an entire video rather than sampling a few frames, providing more complete training data for robotic policy learning.

Key Details

The fine-grained visual analysis capabilities provided by the API include frame-by-frame detection, a 21-keypoint skeleton, left/right hand identity recognition, and per-hand action descriptions. These supervision signals are delivered as sub-task labels for downstream use.

Why Dense Annotation Matters

@lukas_m_ziegler points out that in robotic manipulation tasks, contact between hands and objects changes very frequently; missing frames lead to incomplete training data and degrade training outcomes. @AkshatS07 further explains that for robotic arms or human hands, it is necessary to know exactly what actions the hand is performing and which objects it contacts. Therefore, egocentric data cannot rely on coarse-grained annotation, as robotic policy learning heavily depends on high-quality, fine-grained supervision.

Claimed Results

According to @AkshatS07, incorporating these fine-grained supervision signals into training yields a 77% end-to-end F1 improvement on the WGO-Bench benchmark.

2026-07-10 ~ 2026-07-10 · 5 related posts