Sail Open Sources HTDYM: Analytical Tool for Cheapest Model Deployment Chips
cgarciae88 · x · 2026-08-31
Sail Research has open-sourced HTDYM (How To Deploy Your Model), their internal performance modeling infrastructure for selecting optimal hardware and sharding configurations. Guided by the motto "no bad chips, only bad prices," HTDYM calculates tokens-per-dollar based on model characteristics (e.g., MoE, precision, attention mechanisms) and workload differences (prefill vs. decode). It helps identify undervalued hardware for specific models. The code is available on GitHub with a hosted demo.
More from Infra
- Achieving 200k context on a single 5090 using Q4 KV Cache and SKILL.state — Ok-Shower7286 · 2026-08-31
- Microsoft HotChips presentation reveals bandwidth wall in AI cluster architecture — bookwormengr · 2026-08-31
- Three agent civilizations in 3 months—the marginal token buyer is no longer human — arthurcolle · 2026-08-31
- Musk predicts superhuman digital AI by next year as compute scales — apples_jimmy · 2026-08-31
- Exclusive: SK hynix weighs Intel Foundry for next-gen HBM4E base dies, breaking TSMC dependence — BenBajarin · 2026-08-31
- Parody: The $200/mo user costing OpenAI $14,000/mo — BuildersReadOnAI · 2026-08-31