DeskForge generates 1.2M annotated desktop observations, lifting GUI grounding accuracy by 11.5 points
docling-project · hf · 2026-10-07
docling-project introduces DeskForge, a controllable desktop environment that composes real applications with varied states, layouts, and resolutions to produce dense supervision for computer-use agents.
- The resulting DeskForge-1M corpus contains 1.2M annotated desktop observations with 159.7M element instances, fusing screenshots, accessibility trees, and window geometry.
- Fine-tuning four VLMs on 200K grounding examples improves all of them across five external GUI grounding benchmarks; Qwen3.5-4B gains 11.51 points on ScreenSpot-Pro and 10.11 on OSWorld-G.
- Gains transfer to long-horizon tasks: with a fixed planner, Qwen3.5-4B solves more WebArena-Infinity (31→50 of 119) and OpenApps (3→15 of 100) tasks.
- Code, dataset, and fine-tuned models are open-sourced.
More from coding & agent
- Anthropic designer interview: three designer archetypes and early Claude Code lore — Flomerboy · 2026-10-07
- Coding agents speed up output, but low-level engineering experience still matters, argues AI educator — omarsar0 · 2026-10-07
- Garry Tan Ports Doom to Paul Graham's Bel in 20 Minutes with Opus 5.5 — garrytan · 2026-10-07
- Garry Tan ports Doom to Paul Graham's Bel LISP in 20 minutes using Opus 5.5 — garrytan · 2026-10-07
- L0pht Hacker Chris Wysopal on Securing AI-Written Code: 'Make It Secure' Isn't a Prompt — WeldPond · 2026-10-07
- Designing evals for contract review agents: ten lawyers, ten redlines — graceisford · 2026-10-07