Inkling 2 Small serving details: ~1,070 TPGS agentic throughput at 32K context
vllm_project · x · 2026-11-13
vLLM shares DSpark serving details for Inkling 2 Small: probabilistic drafting hits 3.5 acceptance length at T=1.0. Agentic multi-turn on TP8 at 100 tok/s per user yields 1,070 output TPGS at 32K context and 700 at 128K. Accuracy verified with inkling-verify (NIAH up to 1M, MMMU-Pro, HLE); LoRA and RL (SkyRL) supported.
Related event: vLLM adds Thinking Machines' Inkling-2-Small with 1M context and 1100 tok/s(2 posts)→
More from Infra
- IBM QDC 2026 Unveils Nighthawk r2 Quantum Chip and Fault-Tolerant Dev Tooling — MJBiercuk · 2026-11-13
- Why Jensen Huang Splits From US Frontier Labs on Open Source and Export Controls — morganlai · 2026-11-13
- SGLang-Neuron v0.1.0 open-sourced: SGLang serving stack now runs on AWS Trainium — ying11231 · 2026-11-13
- Nvidia Has $36B in Off-Balance-Sheet Backstops for Neoclouds, Plus $20B in Leases — SumitGup · 2026-11-13
- India's pretraining gap isn't GPU smuggling — it's underinvestment, argues researcher — teortaxesTex · 2026-11-13
- Report: Jensen Huang resigns as Nvidia CEO amid -63% cash conversion, $47B CoreWeave debt — SumitGup · 2026-11-13