Microsoft's 4B Mage-VL-Cowork hits 66% on OSWorld-Verified SOTA; Niels Rogge builds runnable demo on HF Sandboxes

NielsRogge · x · 2026-11-04

Niels Rogge demoed Microsoft's new 4B model Mage-VL-Cowork, which sets a new SOTA of 66% on OSWorld-Verified (excluding Google Drive tasks) for on-device computer use, beating much larger models.

Key details:

Hybrid agent: at each step it decides whether to use the GUI (clicks, keys) or a Bash shell; for structured edits it often just writes Python instead of clicking through menus
Demo architecture: model runs in an HF Space, desktop in an HF Sandbox (isolated VM booting in seconds, Xvfb + XFCE + Firefox + xdotool image, ready in 18s)
Loop: screenshot via sandbox file API → model returns think + action + tool calls → xdotool executes clicks/typing, bash runs in the same box; Gradio UI streams the screen, reasoning, and a red dot where it clicked

A compelling lightweight-model + hybrid-tooling path for on-device agents, with a fully reproducible HF Sandbox setup.

Original post →

Microsoft's 4B Mage-VL-Cowork hits 66% on OSWorld-Verified SOTA; Niels Rogge builds runnable demo on HF Sandboxes

More from coding & agent