Benchmark: pi agent harness outperforms DeepSeek's dsh on cost
solyarisoftware · x · 2026-08-20
DeepAPI benchmarked DeepSeek's official agent harness (dsh) against the pi framework.
- Test Scale: 180 controlled runs across 3 models and identical tasks.
- Cost: dsh was more expensive than pi on 2 out of 3 models.
- Analysis: The difference stemmed from round-trips; dsh required 11 model calls versus pi's 8.5 (on DeepSeek V4 Pro).
- Surprise: The only model where dsh won was Kimi K3, not DeepSeek.
Note: DeepSeek V4 Pro was run on GMICloud fp8, not the official API.
More from coding & agent
- Meta open-sources Muse Spark Open agent model with day-0 TPU support in SGLang — BanghuaZ · 2026-10-16
- Saying "load-bearing" in your prompt makes Opus 5.5 stop holding back — zacharynado · 2026-10-16
- You.com partners with Paperclip to give every agent live web access — PolarBearby · 2026-10-16
- I priced every Grok Bot call: $20 to $200 plan in 5 days, long jobs eat 52% — jonathanmalkin · 2026-10-16
- GitHub is criminally underrated as a free data store, devs argue — blelbach · 2026-10-16
- Claude Desktop doesn't support MCP sampling — here's a workaround for two-way MCP apps — wuj · 2026-10-16