VISTA: A Visual Harness for Reasoning in an Interactive World
Qiushi Han, Keya Hu, Linlu Qiu, Cathy Wu, Kaiming He
cs.AI, cs.CV
2026-10-02
VISTA adds lossless visual memory and inspection to a multimodal model. Opus 5.0 hits 100.00 RHAE on 25 ARC-AGI-3 games with 57.4% fewer actions than first-time humans.
ARC-AGI-3 is a suite of interactive visual games that withhold rules and goals. An agent has to discover how an unfamiliar world works by acting in it. Most existing systems treat this as a language problem: they turn the board into a text grid or a symbolic state, then synthesize programs that serve as an executable world model. That works on grid worlds. It gets harder as scenes become more visual and less easy to write down.
Vision-language models can look at pixels directly, but the usual pipeline encodes each image once and drops the raw frame. Those embeddings can be compressed and lossy. When context fills up, older observations get evicted or replaced with text summaries. At encode time the model also cannot know which pixels will matter later. In a long episode, lost visual evidence does not come back.
VISTA is a visual harness: a mechanical tool layer around a frozen general-purpose multimodal model. Nothing in the harness is trained. It has three parts.
A turn is observe, reason, act. Inspection and notes are optional. Only play spends a game action. The agent keeps two files: GUIDE.md for rules that should survive across levels, WORKING.md as a scratchpad for the current level. Near the context limit it writes a handoff summary and resumes from the current state; notes, visual memory, and the action log persist. Default frames are 512×512 PNGs, an 8× nearest-neighbor upscale of the official 64×64 grid.
Do not freeze every future detail into the first encoding. Keep the pixels and let the model look again.
On the 25 public ARC-AGI-3 games, the organizers' program-free baselines score 13.33 RHAE with GPT-5.6 Sol at max effort and 40.68 with Claude Opus 5.0 at high effort. RHAE (Relative Human Action Efficiency) scores a finished level by action count against first-time humans, up to 115% if the agent uses fewer actions, zero if the level is unfinished, with later levels weighted more and each game capped by the weighted fraction of levels cleared.
VISTA with Opus 5.0 at extra-high effort scores 100.00, using 7,302 actions against the human reference of 17,135, which is 57.4% fewer. GPT-5.6 Sol at max effort scores 99.00. Concurrent program-synthesis systems such as Tycho and Retrodict also land at 99 to 100. Among public results, this is the first vision-based system to reach perfect or near-perfect scores without writing programs.
A GPT-5.6 Sol ablation isolates the parts:
| Setup | RHAE |
| Official text grid | 13.33 |
| PNG images | 47.32 |
| Higher action and time limits | 51.66 |
| Continuous chat plus compaction | 65.82 |
| GUIDE.md and WORKING.md | 70.05 |
| Lossless memory and inspection | 94.10 |
| Pixel readout (full VISTA) | 99.00 |
The jump from 70.05 to 94.10 is memory plus inspection. Image observations versus text grids: 99.00 vs 96.75 RHAE, 9,126 vs 10,614 total actions, 30.7M vs 71.9M tokens per game. A 200K context is the RHAE peak; 100K drops 0.4 points and roughly halves tokens. Cutting the upscale from 8× to 4× removes about 75% of the image tokens; the paper calls the score comparable. With open-weight GLM-5.3 Flash 320B, the official harness scores 1.89 and full VISTA scores 66.93. The same games rendered in 3D still reach 84.12 RHAE, but actions rise from 9,126 to 20,640.
The same harness, barely adapted, goes onto three other suites with GPT-5.6 Sol. On GameWorld (34 browser games, 170 tasks) VISTA records 63.3% success and 79.3% progress, above fresh humans at 40.0% success and the official minimal harness at 20.0%. On AI GameStore (10 games), with the human median scaled to 100, VISTA's geometric mean is 140.3. On 39 BabyVision visual-tracking items, accuracy moves from 41.0% with the official harness to 63.2%.
This is a harness paper. The backbone stays frozen. In interactive visual settings, general multimodal models often fail because they cannot re-attend to pixels they already saw, even when the reasoning is there. The portable piece for practitioners is the interface: archive raw frames, let the model fetch them, keep notes outside the context, hand off when the window fills. The same pattern applies to any UI work that needs long-horizon pixel comparison.
The engineering is incremental. The ablation still shows a roughly 24-point RHAE gap from that increment, which is not noise. The bill is tokens: about 30.7M per game in the default setting, on closed models. The open 320B run only reaches 66.93. A harness can elicit capability. It cannot mint it.
The paper flags two issues. Gains rest on what the base model already knows; VISTA only elicits it. The models were released after these public benchmarks, so training-set leakage is not ruled out. The private ARC-AGI-3 split would be the cleaner test, and it is not reported.
The official Opus number uses high reasoning effort; VISTA runs extra-high. The cleaner head-to-head is GPT-5.6 Sol at max for both: 13.33 vs 99.00. A 100.00 RHAE is a capped score: a level can beat the human action count, but the game score is still limited by how many levels were finished. 3D rendering falls to 84.12 RHAE with more than double the actions, so visual complexity still taxes this memory. BabyVision uses 39 tracking items, not the full 388. Evaluation goes through closed APIs and Claude Code / Codex CLIs; cost and variance are hard to reproduce. The human numbers VISTA beats on GameWorld and AI GameStore are novices or medians.