34 open models self-hosted on a Blackwell cluster for open health evals
keunwoochoi · x · 2026-10-27
The Ortet team (keunwoochoi et al.) self-hosted all 34 open models on Orion, their own NVIDIA Blackwell GPU cluster, to run health evaluations — covering models from Meta, Kimi, DeepSeek, Qwen, Z.ai, Reflection AI and others. Self-hosting gives transparency, control, reproducibility and security; full details are in the blog. Their prior analysis implicitly supports diversity and quality of evaluation items over quantity, and volume 2 of the health eval is in progress.
Related event: Ortet Health Index Ranks 34 Open Models, Meta's Model on Top(4 posts)→
More from Infra
- Celesto launches CI runners nearly 90% cheaper than GitHub Actions, with agent sandboxes — aniketmaurya · 2026-10-27
- AW-0 runs at 230ms on a single RTX 5090, 2.4x faster than autoregressive code planning — trevordarrell · 2026-10-27
- Site tracks US city council topics: data centers rank only behind pickleball courts — zakkohane · 2026-10-27
- MiniMax H3 runs fully on a Galaxy S26: 5-second video with audio in ~7 minutes — BidFunny4862 · 2026-10-27
- Unconventional AI acquires inference chip firm NeuReality in 1,000x efficiency bid — NaveenGRao · 2026-10-27
- Musk: robot RAM demand will dwarf everything else, and 100% of superintelligence compute goes to deep space — elonmusk · 2026-10-27