VLM Run Gateway Launches: One API for Open-Weight OCR, VLM and Vision Models
spillai · x · 2026-10-28
VLM Run launched VLM Run Gateway on Hugging Face — a unified API for running open-weight VLMs, OCR VLMs, and ViT-based vision models, born from production frustrations.
Footguns the team kept hitting:
- Quantized models under the same name: providers serve different quants and serving params behind one model-id; some quants fine on text benchmarks noticeably hurt OCR/small-text/spatial accuracy
- Weak video support: 80%+ of providers on popular routers didn't accept video inputs, and fewer let you control FPS
- Document inference is all pipelining: PDF rasterization, parallel page workers, retries, rate-limits eat dev time
- Leaky abstractions: context limits, max resolution, FPS sampling, quants, and GPU architecture all add quality variance under the same model-id
Goal: one trustworthy entry point for OCR models, VLMs, and ViTs for internal agents and evals, with online demo and uvx CLI commands available.
More from Infra
- EmbeddingGemma 2 runs real-time embeddings on a MacBook CPU alone: 20ms text, 100ms images — generativist · 2026-10-28
- Statistically lossless Qwen3.8-27B GGUF quantizations land on Hugging Face, analysis paywalled — Dany0 · 2026-10-28
- Nebius benchmark: 8 small GPU replicas yield 42.5% more Cosmos video per hour — shashib · 2026-10-28
- Data centers become the new front line as AI, cyber, physical and OT threats converge — ChuckDBrooks · 2026-10-28
- preowned turns macOS built-in Apple models into an OpenAI-compatible local API — pcuenq · 2026-10-28
- 2,100+ tok/s/user for DeepSeek V4.1 Flash on 8× AMD MI355X, 2.8x faster than open-source — JiaZhihao · 2026-10-28