SGLang Summit agenda: LinkedIn serves ads-ranking model at 10K+ QPS under 80ms latency
bgmshana · x · 2026-11-13
Phala co-founder & CTO Hang Yin will speak at SGLang Summit 2026 (Nov 12–13, Fort Mason, San Francisco) on "Production Private Inference Using SGLang and dstack".
The summit site reveals notable sessions:
- Production: Cloudflare on large-scale deployment; LinkedIn serving a small ads-ranking model at 10K+ QPS with sub-80ms scoring latency
- Engine: Meta bringing PyTorch to SGLang; Alibaba extending HiCache for long-running agents
- Agentic engineering: Perplexity's minimalist trainer for trillion-parameter models built with coding agents; Modal using agents to develop and manage inference services
- Hardware: NVIDIA on porting SGLang to Vera Rubin; Google on SGLang-TorchTPU for global production inference
- Also: SGLang Omni interruptible real-time conversation, TorchTitan work from Thinking Machines Lab and Meta, and more
Related event: First SGLang Summit Opens in San Francisco with OpenAI, NVIDIA and Meta(4 posts)→
More from Infra
- Microsoft outage disrupts LA County Sheriff's email and communication systems — Kyrannio · 2026-11-13
- WSJ: Cloud providers fleeing Nvidia as wafer commitments hit Huang like an avalanche — SumitGup · 2026-11-13
- SpaceX builds $24B Starbase supercomputer near Brownsville after clearing 60 years of dumped trash — elonmusk · 2026-11-13
- Dev runs "walking DeepSeek" demo on Mac with SGLang serving inference — ying11231 · 2026-11-13
- Solo devs are outperforming vLLM and SGLang by reworking KV cache and MoE routing themselves — max_paperclips · 2026-11-13
- DeepSeek in a backpack: two Mac Studios hit 220 tok/s fully offline — Stefania_druga · 2026-11-13