Inside RAX's cognitive architecture: exploration turns into versioned robot skill repositories
du_yilun · x · 2026-10-17
This post breaks down RAX's cognitive architecture: the agent uses RGB-D and force observations to build task-relevant 3D models, then searches for actions through the robot's control and kinematics tools, developing its own perception pipelines, manipulation strategies, and execution checks.
As exploration scales, successful decisions become autonomous programs containing the full perception–planning–control loop that run directly; unfamiliar situations hand control back to the agent for further exploration and revision. Memory has two layers: session notes preserve discoveries, while a versioned skill repository stores working code, preconditions, and execution evidence across sessions. Exploration is parallelized across agents and environments.
Quoted context clarifies what is being scaled: the agent's budget of online exploration episodes. The underlying model stays frozen—learning happens in the agent's growing workspace of code, notes, and verified skills.
More from Embodied
- Google Gemma hardware hackathon at AGI House offers $45k prizes, $15k for LiteRT track — agihouse_org · 2026-10-17
- iPhone Duo pre-orders sell out instantly, shipping slips past January 2027 — mitchdeg · 2026-10-17
- P-BFM: unsupervised RL lets Unitree G1 humanoids adapt to rough terrain from flat-ground references — ChongZzZhang · 2026-10-17
- Researcher builds generative machine vision pipeline reconstructing scenes from monocular video — RexDouglass · 2026-10-17
- Starkey Launches Omega AI+ Hearing Aid With Quad DNN Architecture and Sleep Tracking — BrandonSawalich · 2026-10-17
- RAX open-sources everything: zero demos, all experiment sessions included — du_yilun · 2026-10-17