HF engineer open-sources vision-intern: train a $6 object detector instead of using frontier LLMs
mervenoyann · x · 2026-11-17
Hugging Face engineer Merve Noyan open-sourced vision-intern, a Python toolkit from her AI Engineer talk arguing that using frontier LLMs for object detection is a skill issue — use LLMs as "vision interns" and train cheap task-specific models instead.
- The toolkit wraps her favorite models (keypoint detection to zero-shot segmentation) as agent tools: VLM-based dataset labeling, VLM-as-judge annotation evaluation, and training via a few function calls.
- Pipeline: any coding agent orchestrates → small VLMs (Qwen3 series) label images → ensemble voting from judge VLMs (Gemma-4, LFM-VL/d1-Omni) merges verdicts → fine-tune Roboflow rf-detr-large on retained samples.
- Cost: $6 in rented GPUs (Qwen3.5-9B) for a decent detector, free on a local GPU.
- Limitations: no segmentation training yet; very dense instances may not work.
More from coding & agent
- Daniel Miessler replaced his Apple TV YouTube app with an AI-built custom client — RealGeneKim · 2026-11-17
- Google paper: Self-Improvement Memory boosts continual recursive self-improvement in agent harnesses — omarsar0 · 2026-11-17
- Stop using frontier LLMs for vision: a $6 detector trainer, distilled into agent tools — mervenoyann · 2026-11-17
- Hacking a contour dial into a Krita AI diffusion controller with Claude — MissionRoutine · 2026-11-17
- Hacking an Unused Controller Into a Krita AI Diffusion Dial With Claude — Dacrikka · 2026-11-17
- Chrome lets sites self-label AI safety for agents — who warrants the claim? — YvesMulkers · 2026-11-17