Human-Centric Intelligence in the Era of Foundation Models: A Survey
Yang Chen, Tianqi Wang, Xiaorui Jiang, Yilei Man, Yihua Shao, Mengyuan Liu, Zhi Chen, Xiaofeng Cao, Qibin Zhao, Chi Harold Liu, Albert Y. Zomaya, Nicu Sebe, Jingren Zhou, Dacheng Tao, Song Guo, Jingcai Guo
cs.CV
2026-08-18
A six-level taxonomy maps human-centric AI from appearance and geometry to motion, interaction, world models, and embodiment; the gap is transfer, physics, and deployment.
Human-centric research has been sliced by task for years: one survey for pose, one for action recognition, one for motion generation, another for human-object interaction. Each slice is detailed. The joints between slices are not. Foundation models made reusable backbones, multimodal interfaces, and cross-task transfer the default story in vision and language. Human-centric work has not finished that same transition. A prior survey of human-centric foundation models still mostly looks at the person in isolation, with a thinner literature cut. This paper, led by The Hong Kong Polytechnic University with Peking University, Alibaba, NTU and others, tries to draw a full-spectrum map for the foundation-model era.
The scope is explicit. In: methods that treat humans as observable subjects, dynamic actors, or situated agents. Out: HCI, human-oriented language modeling, and policy writing about human-centered AI.
The taxonomy expands representational scope across six levels and three viewpoints.
Observable subjects. Visual appearance asks what a person looks like: generalist perception, identity, controllable generation. Sapiens trains ViTs on 300 million human images; Sapiens2 scales that to one billion images and 0.4B–5B parameters. Spatial geometry asks how the body occupies 3D space: pose and mesh recovery, renderable avatars.
Dynamic actors. Kinematic dynamics tracks how body state evolves over time: motion understanding and generation, human video animation. Interaction modeling treats objects, scenes, and other people as part of the problem. Contact, affordance, and social coordination are the human-specific grounding.
Situated agents. World simulation lets human action and world state co-evolve; egocentric video is the interface. Embodied agency turns human experience into executable control: humanoid policies, skill transfer from humans to robots.
The methodological substrate is three layers. Data comes in five families: visual imaging, spatial structure, sensorimotor, wireless and ranging, linguistic and acoustic. Architectures fall into four paradigms: single-step mapping, sequential factorization (autoregressive), iterative generation (diffusion or flow), and hybrids. Training has seven mechanisms, from training from scratch through LoRA, instruction tuning, reward tuning, distillation, and task-head tuning. Inference-time tools are semantic augmentation, guided sampling, iterative refinement, preference selection, and retrieval. Methods that span levels are assigned by primary modeling target and evaluation objective.
This is a survey. There is no new bake-off. The result is an organization, not a leaderboard.
Across all six levels the shared foundation-era move is from specialist estimators to reusable priors plus a unified interface. On appearance, Instruct-ReID recasts re-identification settings as instruction-guided retrieval, and ChatReID uses a vision-language model for fine-grained matching. On geometry, feed-forward reconstruction and scalable human priors pack several functions into one system. Motion and animation are being tied together by large corpora, language-aligned interfaces, and pretrained generative backbones. World models are shifting from passive future synthesis toward action-conditioned simulators. On embodiment, generalist humanoid policies and latent-action pretraining sketch a path across bodies; the embodiment gap remains.
Datasets, benchmarks, and metrics are tabulated. The authors' claim is sharp: the shortage is not the number of metrics, it is a protocol that connects them. High scores on separate tasks do not show that a model transfers across human contexts, or that several capabilities work inside one process.
Six open problems follow: scalable and traceable human data (synthetic data also scales the generator's artifacts and biases); a unified human-centric foundation model; putting physical constraints inside the representation rather than as a post-hoc penalty; human-centric world models with closed-loop action intelligence; a benchmark suite for generalization and composition; modular systems that can actually be deployed.
For anyone shipping human-related products, the value is an index and a coordinate system. A new paper can be placed on one of six levels, one data family, and one architecture class before anyone decides to follow it. Human-specialized pretraining at Sapiens scale, instruction-shaped retrieval, and the warning that visual plausibility is not physics, all travel into model selection.
It will not teach a reproduction. The contribution is joining fragments into a spectrum. Given the coverage and the exclusion list, the judgment that human-centric intelligence has not finished the foundation-model transition that language models already made is fair.
The authors put the gaps in Section 8. Unified models still sit on restricted task groups or a single modality. Vision-language priors can generate plausible human behavior that violates physics. Egocentric video is plentiful; observed behavior does not become robot control. Missing evaluation protocols mean the surveyed methods are hard to compare sideways.
Two further discounts. Calling this the first full-spectrum survey is the authors' own claim; they already cite the HOI survey and a human-centric foundation-model survey, so the title depends on how "full spectrum" is defined. Methods that do both mesh recovery and interaction get pinned to one primary level, and a reader who only skims the taxonomy will miss the crossover. The GitHub collection is described as living; the paper body is an August 2026 snapshot.