Together rewrites its inference platform after serving 400 trillion tokens a month
graceisford · x · 2026-07-24
Together says it has rewritten its inference platform from the ground up after learning how AI-native teams and enterprises use open and custom models in mission-critical production.
What changed
- The new platform is now live.
- It is designed for running open models in production with full control.
- Teams can test every change on live traffic before users see it.
- The company says it serves 400 trillion tokens per month, which shaped the redesign.
The focus here is infrastructure: safer production workflows, controlled deployments, and operational lessons from large-scale inference traffic.
Related event: Together.ai Launches New Open Model Inference Platform(4 posts)→
More from Infra
- DeepSpeed shows AutoEP/AutoTP/AutoSP parallelism gains, hosts office hours — PyTorch · 2026-10-27
- Cloudflare Durable Objects validated as top architecture for AI agent WebSocket servers — tristanbob · 2026-10-27
- Gary Marcus: With Q3 data added, Anthropic's compute-efficiency growth shows deceleration — GaryMarcus · 2026-10-27
- SpaceX locks in $40B GPU financing deal with Apollo, the largest ever committed — TechCrunch AI · 2026-10-27
- Inference is just Kubernetes with a multi-trillion-dollar market, and the communities overlap — matt_slotnick · 2026-10-27
- vLLM eyes Rust migration as inference engines race to rewrite the stack — AccBalanced · 2026-10-27