Hark on scaling multimodal data curation with SGLang and KubeRay, plus faster inference-worker startup
tianyin_xu · x · 2026-11-05
In talk #8 of LMSYS's series, Kai-Hsun Chen and Cindy Zhang of Hark Labs break down how they scale multimodal data curation with SGLang and KubeRay.
Key topics: large-scale multimodal data curation pipelines, Ray-based GPU orchestration, and techniques to speed up inference-worker startup. Useful reference for teams running multimodal training-data pipelines and inference infrastructure.
More from Infra
- Dev: skip any GPU under 256GB for local AI, prices are about to fall — AIFlow_ML · 2026-11-05
- int8-vllm fork rewrites serving stack, hits 1000+ tok/s on 4x MI100 — 1ncehost · 2026-11-05
- US economy runs on just 18% electricity, transport only 1% electrified — NinaDSchick · 2026-11-05
- AI industry's new defaults: 15-25% of planned compute at risk, Anthropic IPO haircut — sytelus · 2026-11-05
- hxiao: inference overfitting is real, Together Compiler hillclimbs TPS on task token distributions — simran_s_arora · 2026-11-05
- SGLang Summit Lands in SF Nov 12-13: 2,000+ Builders, Production Lessons From NVIDIA, AMD, Alibaba and More — ying11231 · 2026-11-05