Qwen3-VL-2B fine-tuned into a computer-use agent that does cell segmentation in a headless labeling app
secemp9 · x · 2026-09-10
Inspired by Astra's computer-use abilities, a developer fine-tuned Qwen3-VL-2B into a specialized computer-use agent: it takes screenshots as input and outputs commands to draw polygons, zoom, and navigate — autonomously performing instance segmentation of cells in a headless labeling app. All navigation and labeling in the demo video is done by Qwen itself.
secemp9's take: the ideal model should be good at both tool call-based vision and native vision, with each generalizing to the other — and even better if the model knows which to use at each step while still obeying explicit user prompts like "only use internal vision" or "use only tools". The route of training specialized computer-use agents clearly works.
More from coding & agent
- Dev compares Astra vs ChatGPT coding style: 'a programmer from 1980' vs clean minimal — cnakazawa · 2026-09-10
- Diffing two agent runs yields 40 real differences — which 2 matter? — Sensitive-Parsnip-12 · 2026-09-10
- SkillsGate: open-source visual manager to install AI agent skills across 20+ tools — tom_doerr · 2026-09-10
- AI coded 300 tasks, claimed all done — 11 of first 28 were broken — InspectorSorry85 · 2026-09-10
- DeepSeek open-sources deepseek-recipe to ease deploying V4.1 Flash and future models — teortaxesTex · 2026-09-10
- Rysy: open-source agent that profiles people before you email them — Acceptable-Swing1619 · 2026-09-10