EarthRecord benchmark tests 12 multimodal agents on four decades of raw satellite imagery
AIEyesTech · hf · 2026-11-11
Researchers from HKUST(GZ), HKU and collaborators introduce EarthRecord, a benchmark measuring whether multimodal agents can turn decades of raw satellite imagery into reliable land-change analysis.
- Built on raw multi-sensor archives (Landsat, MODIS, VIIRS, Sentinel-2) for 56 sites across four decades; cloud masking and cross-sensor harmonization are left to the agent. It poses 238 land-change questions across ten analysis tasks.
- Evaluations of 12 multimodal agents show leading agents complete most analyses end to end but falter when answers require spatial mapping from imagery.
- Two controls probe what answers rest on: a closed-book setting and instrument pairs where one satellite's data is altered while nothing changes on the ground. The record moves answers from prior knowledge toward pixels, but not yet to real change—when one satellite's data is altered, half of the correct answers follow the alteration.
Paper, code and dataset are openly available.
More from Research
- Laser-as-a-sensor: monolithic VCSEL oxygen detector shrinks sampling volume to subnanoliter — jwt0625 · 2026-11-11
- AI×Antibodies hackathon in Oslo offers JapanFold, Anthropic credits, $20K lab validation — DavidBennett__ · 2026-11-11
- Which benchmarks do humans still win against frontier AI? Reddit debates AGI criteria — ImproveOurWorld · 2026-11-11
- WaveGen: codec-free end-to-end diffusion hits SOTA text-to-speech via Spectral Forcing — Ajou · 2026-11-11
- Berkeley's Social Humanoid teaches Unitree G1 socially responsive interaction — berkeley_ai · 2026-11-11
- Robotics is not solved: frontier models are ~6 orders of magnitude off real assembly lines — ZeeshanZiaML · 2026-11-11