Tencent's HiSTEVE 1.0: hierarchical VLA agent plays Minecraft with native keyboard and mouse, hits 85% SOTA
_akhaliq · x · 2026-10-26
A Tencent team published HiSTEVE 1.0, a hierarchical Vision-Language-Action agent that autonomously completes long-horizon tasks in Minecraft using only visual observations and native keyboard-and-mouse controls — no privileged game states or environment APIs.
Key design:
- Orchestration level: a capability-aware harness with structured memory for progress tracking and a VLM planner that selects subtasks within the VLA's capabilities; a policy-coupled completion detector signals subtask completion
- Control level: a single VLA executes all subtasks with vision-only perception and native mouse/keyboard actions; regime-specific RMSNorm branches and separate mouse-movement token embeddings mitigate interference between GUI and non-GUI interactions
- Training: world knowledge injection, imitation learning, and RL with environment feedback
Results: state-of-the-art 85.0% on MineStudio Bench, 37.4% average success on OpenHA, and long-horizon feats like crafting an iron pickaxe from scratch.
Related event: HiSTEVE 1.0: Hierarchical VLA Agent Plays Minecraft with Keyboard and Mouse(2 posts)→
More from Embodied
- Humanoid walking is continuous correction — watch recovery, not locomotion, in demos — Olivier__OG · 2026-10-26
- CircuitHub unveils The Grid, a factory-scale robot making PCBs in the US twice as fast as Shenzhen — garrytan · 2026-10-26
- AI researchers guessed robots wouldn't outrun humans until 2034 — it already happened — ben_j_todd · 2026-10-26
- Shanghai AI Lab-led team releases 100-hour egocentric wearable dataset for humanoid training — ChuckDBrooks · 2026-10-26
- RealSense D585 Pro AI-Powered 3D Stereo Depth Camera Unboxed for Physical AI — chrismatthieu · 2026-10-26
- Physical world agent swarm demo shared on X — pduan · 2026-10-26