Seven local open VLMs were tested as a second reader, and Qwen3-VL 4B won
MaziyarPanahi · x · 2026-07-22
A local evaluation of seven small open VLMs found that second-reader behavior matters more than simple text reading.
- Qwen3-VL 4B got all 7 identifiers right in 6.9s with zero false alarms and was picked for the role.
- Gemma 4 E2B reached 6/7.
- Qwen3-VL 2B was fastest but only found 4/7.
- NVIDIA LocateAnything-3B drew perfect boxes but failed to judge what was actually personal.
- LFM2.5-VL 1.6B and InternVL3 2B read the patient name correctly, then incorrectly marked it as not personal.
The takeaway: detection is easy; judgment is the hard part. The poster also links an Apache 2.0 “Reader 1” and says both models are on Hugging Face.
Related event: Qwen3-VL Beats 6 Open-Source VLMs in Medical Image Desensitization Test(3 posts)→
More from Research
- Open-source decision models sold their own base in Red Alert 2 — then learned until no human could beat them — heathercmiller · 2026-10-27
- DeepMind paper: LLMs can't sample randomly on their own — pass random numbers from code instead — rohanpaul_ai · 2026-10-27
- World Model Assigns Weight and Friction to Objects, Then Simulates with a Physics Engine — jonstephens85 · 2026-10-27
- Token Collapse: Study Finds Graph Soft Prompting Tokens in LLMs Fail to Encode Structure — Sapienza · 2026-10-27
- Mathathon 2.0 launches with Terence Tao's SAIR, $1.5M in compute for teams — _sathvikr · 2026-10-27
- The 'sweet lesson' is real: two kinds of ML problems need opposite strategies — mike64_t · 2026-10-27