Running 500B+ Models Locally? Seeking Prefill/Decode Benchmarks
HankYeomans · x · 2026-10-31
A nettermina thread (RT'd by Hank Yeomans) calls on people who own enough Spark devices and Mac Ultras to run 500B+ parameter models locally to share their prefill and decode speed data points, including model and quantization choices, ideally at 32k/64k/128k context lengths.
The author notes such firsthand data is rare, making this a useful signal of demand for reproducible benchmarks on consumer/local hardware clusters.
「Infra」频道最新
- CPU 跑 agent、GPU 按需调用：一场生成式媒体 agent 实战工作坊 — Al_Grigor · 2026-11-01
- 亚马逊被曝封存 2-3 年役龄的 H100/H200,谨慎 GPU 折旧陷阱 — SumitGup · 2026-10-31
- 伊拉克网友收二手 Threadripper 工作站，求 4-8 卡 3090 LLM 服务器建议 — Ammoryyy · 2026-10-31
- Altman 称个人电脑囤 RAM 很浪费，网友晒 64GB 跑 Hello World 吐槽 — SuB8u · 2026-10-31
- xAI 概念图:月球电磁质量投射器年发 1000+ TW AI 卫星算力 — XFreeze · 2026-10-31
- OpenCode 单机架供给 10T Token,多数创业公司算力需求远低于此 — JoshPurtell · 2026-10-31