Cold Start Benchmark: 244 GiB Model Loads in 154 Seconds
QuixiAI · x · 2026-07-30
QuixiAI shared performance metrics for cold starting a massive 244 GiB model. The test reveals a load time of approximately 154 seconds on specific hardware setups, noted as being even faster than llama.cpp's loading times.
More from Infra
- SK Hynix Earnings Analysis: AI Memory Demand Strong, Market Overreacts to Oversupply — tengyanAI · 2026-07-30
- Single 8x 5090 Rig Hits 167k tokens/s Training Throughput, Beating DDP — jon_durbin · 2026-07-30
- Buildcleaner reclaims 443GB of disk space by cleaning build artifacts, free and open-source MIT — jasonkneen · 2026-07-30
- LLM Inference Costs Drop Below $3 with B200s, Yet API Prices Stay High — AccBalanced · 2026-07-30
- Bought GPUs to Escape API Fees, Realized a Single RTX 5090 Is Enough — Ok-Shower7286 · 2026-07-30
- Yann LeCun and Others Discuss: LLMs are the New Compilers, Performance is a Function of Compute — yisongyue · 2026-07-30