YouJiacheng: post-training needs high tps for long-tail rollouts
YouJiacheng · x · 2026-11-07
Responding to a discussion on inference throughput, YouJiacheng points out that post-training needs high tokens/sec because of long-tail rollouts — a few unusually long generation trajectories dominate the throughput requirement, a key constraint for RL post-training infrastructure.
More from Infra
- Open-source fork runs 47GiB MoE on 2019 Radeon VII with 49% faster decode — milpster · 2026-11-07
- SmolLM-135M running in pure SQLite at ~40 seconds per token — cneuralnetwork · 2026-11-07
- NUS professor mocks Jensen Huang: he doesn't need a Chinese spy to sell chips to China — pmddomingos · 2026-11-07
- Anthropic's amended S-1 admits mounting losses amid $110B in annual compute commitments — GaryMarcus · 2026-11-07
- Xepak: declarative local builder for REST, RPC, MCP and RAG servers that agents can write DSL for — rumatoest · 2026-11-07
- US AI scale now hinges on coordinating power, components, agentic workflows and humanoid pilots — ingliguori · 2026-11-07