Tsinghua's TokenRouter boosts LLM serving throughput 64x

Tsinghua's nics-efc lab introduced TokenRouter, a serving system for token-level small/large model routing that achieves up to 64x higher decode throughput than existing single-model inference systems.

2026-10-09 ~ 2026-10-10 · 2 related posts

• TokenRouter Serving System Boosts Token-Level LLM Routing Throughput up to 64x — nics-efc · 2026-10-09
• Tsinghua's TokenRouter: Token-Level LLM Routing Hits Up to 64.15X Serving Throughput — rohanpaul_ai · 2026-10-10