LLM provider TTFT monitoring reveals 7x latency spread for the same model
lluisantoni · reddit · 2026-10-22
A developer at Kottos AI shares how they monitor LLM provider queues by probing each provider every minute with short prompts, using TTFT as a proxy for RTT plus queue time.
For DeepSeek V4.1 Flash, 24h median TTFT ranges from 112ms at Baseten ($1.20/M) and 162ms at Fireworks US ($1.80/M) to 311ms at DeepInfra ($0.42/M) and 755ms at Fireworks elsewhere — a 7x spread for the same model, with latency spikes even within a single provider.
Raw TTFT data for 19 provider-model pairs (including Claude Opus 5.5 and GPT-6 Sol) is published at data.kottos.ai/probe.json, updated every minute; RTT measurements stay private.
More from Infra
- GitHub moves 99.99999% of repos to Azure; per-reference quorums fix 25ms latency wall — rseroter · 2026-10-22
- Flint model weights hit HuggingFace, runs on DGX Spark, RTX GPUs and M4 Pro — testingcatalog · 2026-10-22
- ABB ships RobotStudio HyperReality on NVIDIA Omniverse; Absolute Accuracy calibration is key — lukas_m_ziegler · 2026-10-22
- Flappy Airplanes raises $1B Series A co-led by Index and Lux for new GPU primitives — gabriel1 · 2026-10-22
- Researcher: the final megakernel isn't a kernel but a transformer in an endless loop — DimitrisPapail · 2026-10-22
- SPCX earnings call highlights 'computer per watt' metric as AI value shifts down the stack — mobileraj · 2026-10-22