miHoYo's Gamer: a pixel-to-keystroke agent finishing 83.6% of Honkai: Star Rail side quests

Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent

Run Luo, Kuan Zhang, Qi Deng, Yukun Chen, Zhihao Yang, Yufei Zhang, Renrong Hu, Yi Zhang, Guiyu Ma, Tianyu Fan, Longze Chen, Hao Sun, Jiaming Li, Jiaming Yang, Xiaohao Liu, Tao Tan, Minzheng Wang, Hangtian Jia, Shuai Yuan, Lvdi Wang, Xiao Liu, Chao Ma, Yuting Zou, Yinhe Zheng, Xiaobo Xia, Dawei Jiang, Shaoqing Ren

cs.CL, cs.AI, cs.CV

2026-11-14

miHoYo's Gamer, trained on tens of thousands of hours of keyboard-mouse gameplay, clears 83.6% of Honkai: Star Rail side quests; the best general baseline reaches 45.5%.

What problem this solves

Game AI's landmark results (Atari, Minecraft, StarCraft) all rested on privileged access: internal game states, tailored action spaces, dedicated APIs. Modern commercial games expose none of that, so a generalist agent has to play like a human, reading raw pixels and emitting keyboard-mouse events. General-purpose multimodal models understand the screen well enough, but they never learned low-level control, and they are slow. The game does not wait: in the paper's measurements, switching general baselines from time-paused to real-time evaluation costs 17 to 35 points on average, with a worst single-case drop of 73 points on MCU.

The second problem is horizon. A single Honkai: Star Rail side quest carries 30 to 55 milestones with budgets up to 100 minutes, exceeding any context window, with rewards that only land at the end and errors that accumulate. miHoYo's Honkai AI team built Gamer to answer both: one pixel-to-keystroke action space, plus a training and serving stack that survives long horizons and real-time constraints.

Method

The base is Qwen3-VL-8B (they also trained a CPT checkpoint on Qwen3.5-35B-A3B MoE), in three stages.

CPT for control. Observations at 5 Hz; each 200 ms interval carries six 33 ms action blocks, an effective 30 Hz control rate. Mouse deltas plus key sets are serialized in the model's existing vocabulary, with no extra output head. Each game ships a control specification describing what every key does. The corpus is about 2.4K hours of curated gameplay (VPT, D2E, OpenP2P, plus their own HSR recordings) mixed with 15% general multimodal data. Two debiasing designs matter. Random key remapping applies a random bijection over rebindable keys within each trajectory, so the model learns 'move left' rather than 'the A key'. Repeat-aware loss decay handles the fact that over 85% of action steps repeat the previous one: consecutive repeats are down-weighted exponentially (γ = 0.9), concentrating supervision on behavior transitions like stopping and turning.

Sparse-thinking SFT. From 68K sessions and 11K hours of HSR recordings, 1K hours survive filtering; GPT-5.6-Luna then annotates 1.2M decision-point thoughts, 223K memory summaries, and 63K task segments. Instructions come at three levels: local operations, subgoals, complete tasks. Sparseness is the point. Routine movement emits an action directly; only route corrections and target switches get explicit reasoning, decided inside the same autoregressive pass with no external classifier. Memory works by summary: history about to leave the context is compressed into a first-person progress note before eviction, and the oldest summaries are dropped when the budget overflows. During training, each step sees only what would be available at inference.

TA-GRPO. Memory compaction slices over-long rollouts into finite segments linked by those summaries, so a 100-minute task never has to fit in one context. Segments are scored by engine-verified outcomes (the game engine is instrumented to expose task state; GM commands give reproducible initialization), normalized within groups of 8 rollouts, and credit is distributed with a temporal decay that weights steps closer to the verification point higher. No critic, no process reward model. The sandbox runs 32 parallel Windows game instances served by a sharded pool on 64 H200 GPUs.

Streaming-vLLM. Stateful KV-cache reuse across turns (with incremental RoPE phase correction after compaction), FSM-drafted structural tokens verified in BF16, and early dispatch of the first action block while decoding continues. Dense and MoE backbones on both A800 and H200 are supported.

Results

Two held-out HSR side quests (evaluation tasks never seen in training):

ModelOpportunity Makes the ThiefBestial Ferocity
Best general baseline23.3 (DeepSeek-V4-Flash-Vision)45.5 (Gemini-3.8-Flash)
Qwen3-VL-8B base0.013.9
Gamer-CPT34.447.9
Gamer-SFT72.269.1
Gamer-RL80.083.6

After RL, combat and UI interaction hit 100% on both quests, dialogue reaches 88.1%/100%, navigation 69.1%/80.2%. On four public benchmarks (MCU, ViZDoom, OmniGameArena, GameWorld) under real-time evaluation, the 35B-A3B MoE checkpoint averages 31.5 versus 17.6 for GPT-5.6-Sol, and scores 65.3 on MCU against a baseline best of 8.7. The RT-versus-TP gap is the telling number: most baselines lose ground, while the MoE checkpoint gains +3.4, because fast inference lets actions catch up with the game state. Low latency is itself part of the capability. On 40 minigame instances across seven types, RL reaches a 49.7% average versus 17.1% for GPT-6.1-Sol.

The 120-configuration scaling study (45K H200 GPU-hours) also delivers the cold water: within-distribution fit is excellent (R² = 0.995), but for distant unseen games the fitted data exponent is b ≈ 0. More hours stop helping.

Why it matters

The value is the full pipeline rather than any single idea: repeat-aware reweighting, sparse thinking, summary-based memory, segment-level GRPO, streaming serving. Each piece addresses a concrete failure mode of long-horizon real-time control, and none of it is game-specific; long-horizon robot stacks face the same problems. For a studio, the engine-instrumented sandbox serves RL, evaluation, and automated regression testing through one interface, and the paper itself lists gameplay-testing automation as a core use, a nearer-term product than beating human players. Keyboard-mouse trajectories are also far cheaper than robot data, and public corpora such as VPT and OpenP2P already exist. The inference engine and CPT checkpoints are open-sourced.

Limitations

Admitted by the authors: CPT gains do not extend to distant unseen games; the HSR evaluation tests new tasks inside a familiar game, not zero-shot transfer; segments within a GRPO group do not start from identical states; and the temporal weights are a prior, not evidence that later actions matter more.

Readers should discount further. RL is not uniformly positive on minigames: Hexanexus drops from 76.2 to 33.3 and Janus' Maze from 52.4 to 23.8; timing-sensitive control and fine spatial reasoning (Abacus Circuitry at 22.2%) remain weak, and the three repair puzzles in Bestial Ferocity sit at 11.1% after RL. All three levels of SFT instructions come from a single game, so task-level cross-game generalization rests on indirect CPT evidence. The 'comparable to skilled players' claim never gets a human baseline number. Annotation quality depends on GPT-5.6-Luna, with no reported human audit error rate.

Terms

Source

What people are saying

Related papers

All paper explainers