In-Context Learning for Robots: Methods and Applications
Haojian Huang, Zexi Li, Junhao Guo, Yehang Zhang, Wenxuan Peng, Bohan Zhou, Weilin Ruan, Leyi Wu, Chenxu Wang, Jianchong Su, Binghui Xie, Wosong Chen, Yingjie Xu, Tianhao Zhou, Suzeyu Chen, Pukun Zhao, Jiaqi He, Xinyi Li, Runze Li, Peiran Dong, Shaoxiang Dang, Jing Huang, Yingbing Chen, Yifan Chang, Tianyi Zhang, Shiyuan Deng, Haozhi Wang, Yangkai Wei, Wenqian Li, Han Yang, Kaiwen Zhou, Huaping Liu, James Cheng, Rui Shao, Donglin Wang, Yaochu Jin, Jianye Hao, Ying-Cong Chen, Yinchuan Li
cs.RO, cs.LG
2026-09-29
Survey defines robot ICL as adapting behavior via demos and interaction with weights frozen; 412 references split into four method families, 72% from 2024 on.
A robot can be pretrained to grasp nearly anything and still not know which part goes in the box first, which steps are already done, or how tight a new fit should be. Broad pretraining supplies motor competence, not the decision-relevant facts of the current job. There are two ways to close the gap: fine-tuning writes new evidence into weights and pays a training cost on every task change, while in-context learning (ICL) puts the evidence in the input and leaves the weights untouched. This 39-author survey from eight institutions studies the second route: how demonstrations, instructions, corrections, and interaction history change deployed behavior with neural parameters frozen, and whether the change survives when objects, environments, or execution conditions change.
The field is young and fast-moving: 72.1% of the 412 cited references were first released in 2024 or later, 197 of them in the first nine months of 2026. Earlier surveys organized learning from demonstration by teaching interface, or covered one slice of ICL. This one picks a different axis: the intermediate that carries evidence into execution.
Three layers. First, six learning horizons rank where learning changes the system: S1 explicit control programming, S2 neural policy learning, S3 history-based adaptation, S4 in-context task learning, S5 physical recursive self-improvement, S6 collective knowledge evolution. S3 and S4 are the survey's subject, both with weights frozen at deployment; the authors say plainly that S5 and S6 are research objectives, not realized capabilities.
Second, one shared interface: action blocks At πθ(·|ht, Ct) with θ fixed. Adaptation can live in five places: supplied evidence, recurrent state, an external memory archive, test-time fast weights, or weights retained across tasks. The first three are ICL; the last two are parameter adaptation. This yields evaluation controls directly: remove the evidence, clear the memory, restore the weights, and attribute gains piece by piece.
Third, the main taxonomy sorts methods by the intermediate execution consumes:
The intermediate matters because one demonstration can serve as policy conditioning, a motion reference, a constraint on a predicted future, or a skill request. Each family carries its own transfer assumption: action inference must preserve the taught distinction, geometric transfer assumes the matched interaction stays applicable, world-model control assumes the predicted evolution is physically realizable, and skill execution assumes the available skills preserve the task constraints. Two mechanisms cut across families: correspondence (geometric, semantic, and temporal alignment that binds evidence to the scene) and memory (keeping evidence available over time).
Corpus composition through 29 September 2026:
| Slice | Count | Share |
| All references | 412 | 72.1% first released 2024 or later |
| Method and comparison subset | 260 | 100% |
| Context-conditioned policies | 123 | 47.3% |
| Skill- and agent-based execution | 91 | 35.0% |
| Geometric demonstration transfer | 24 | 9.2% |
| World-model-based control | 22 | 8.5% |
Two families absorb roughly 82% of method work. Three comparisons stand out. In REGENT's unseen-MuJoCo-embodiment test, retrieve-and-play beat the frozen contextual transformer: under distribution shift, a learned interpreter can transfer worse than replaying archived actions. KAT found contextual inference competitive with few examples while a trained policy wins with more, and the crossover depends on representation and mechanism. MT3 stacks language filtering, geometric retrieval, alignment, and replay into a repertoire of 1,000 single-interaction tasks from under 24 hours of human demonstration, with no policy retraining.
The survey also pushes an evaluation discipline: measure responsiveness to teaching, physical transfer, and gains from retained experience separately, instead of collapsing them into one success rate.
For practitioners in embodied AI, this is a map plus a checklist. Given any "robot learns from a demo at deployment" system, the four-family coordinates plus correspondence and memory locate its transfer assumption and its most likely failure point. The remove-evidence / clear-memory / restore-weights interventions work both for benchmark design and for reviewing papers. Where task information lives, in context or in weights, determines acquisition cost, persistence, and reset behavior, and matters more at selection time than a single success rate. The companion GitHub list awesome-robots-icl follows the same taxonomy and works as a reading index. To be clear, this is a survey: no new method, no new experimental numbers; the output is structure.
The authors draw boundaries themselves: a mechanism-oriented narrative synthesis, not a systematic meta-analysis; S5 and S6 are aspirational; hybrids are classified by their deployed branch, so calls like placing GPT-Policy under contextual tool use involve judgment. Coverage stops at 29 September 2026 in a field producing close to 200 papers a year, so the map has a shelf life. One concern from reading it: the contact email and both corresponding authors' primary affiliation are Knowin AI, and systems described in the text (GLOW, whose description mentions KnowinWorld; Robo-Harness K1) sit in prominent taxonomy positions. The family-share statistics reflect the authors' coding choices and deserve a discount when quoted as an objective field-wide distribution.