LithosAI launches Kimi K3 with 800 tokens/sec inference speed on standard GPUs
Tim_Dettmers · x · 2026-08-20
LithosAI has announced its first pricing tiers and API launch, targeting ultra-fast agentic inference. The Kimi K3 model achieves over 800 tokens per second per user on standard GPUs while maintaining full model quality, claiming to be up to 16x faster than major providers.
Key Features:
- Extreme Speed: Tested at 800+ tokens/sec/user on a single 8×B300 node.
- Agent Optimized: Low latency across the entire agent loop (plan, call tools, observe, retry).
- High Compatibility: Offers OpenAI and Anthropic compatible APIs; works with Claude Code, Codex, and other harnesses by changing just the base URL.
- Model Support: Also serves GLM 5.3, Qwen 3.8, DeepSeek V4, and more.
Related event: LithosAI launches Kimi K3 with 800+ tokens/s inference(3 posts)→
More from Infra
- FT warns of 'Darwin moment' for AI borrowers as CoreWeave bonds sink to distress levels — firstadopter · 2026-10-16
- Phone as interface, home servers as the lab: a two-workflow local AI setup — EAccelerate_42 · 2026-10-16
- "A datacenter is a bank": CoreWeave drama revives calls for算力 supervision — arthurcolle · 2026-10-16
- DOT-MoE: converting dense models into sparse MoE via optimal transport — burkov · 2026-10-16
- Stanford CS336: A 7B model needs 14GB for weights but 84GB to train — stanfordnlp · 2026-10-16
- New NVFP4 trick: compressing block scales saves 45-48% per expert matrix, bit-exact — TheZachMueller · 2026-10-16