HF's merveenoyan open-sources vision-intern: multi-role VLM pipeline for auto-labeling datasets
mervenoyann · x · 2026-09-07
Hugging Face's merveenoyan open-sourced vision-intern (experimental), a Python toolkit that labels datasets with VLMs, evaluates annotations via VLM-as-judge, and trains object detection models—from a few function calls.
The recommended architecture keeps model roles separate (and on separate processes/endpoints—don't collapse into one model):
- Orchestrator: 10-12B thinking model drives the workflow, babysits runs, fixes errors; never labels or judges itself (agent/subagent).
- Labeller: 7-8B VLM generates detections, remote (HF Inference Providers) or local, $0.5/1k images.
- Judges: several 2-5B VLMs score/verify the labeller's output, with ensemble/voting signal.
Repo includes examples, jobs, tools, and workflows; the author plans to refine it soon for agentic use.
More from coding & agent
- Building a neutral Codex-class harness requires running the original inner harness, dev says — joshalbrecht · 2026-09-07
- Dev Builds Timestamped Summary Site for All MLOps Community Talks, With an MCP Server — strickvl · 2026-09-07
- Shipping an Obsidian plugin with GPT-6 Astra: scan limits, SVG icons, review delays — vista8 · 2026-09-07
- Grok Build 1.0.22 ships subagent messaging, first-party MCP server and safer Auto mode — mark_k · 2026-09-07
- Agents on a 16GB MacBook Air M5: Qwen 3.5-9B 4-bit can't yet build a working app — Fluid-Author-9566 · 2026-09-07
- Remastering old i2v videos with MiniMax H3: a depth-map + reference workflow — WaitAcademic1669 · 2026-09-07