Local agent benchmark: 2nd agent gives 1.5x throughput, the 4th only hurts
AIForOver50Plus · reddit · 2026-08-20
A Reddit user stress-tested multi-agent concurrency on a MacBook Pro M3 Max (128GB unified memory) with 4-bit Qwen3 27B, turning three community predictions into experiments:
- The second agent helps, the fourth doesn't: aggregate throughput rose from 16.6 to 20.8 tokens/s from 1 to 2 agents, then flattened and drifted down at 4 and 8. Per-agent decode collapsed from 17.4 to 3.9 tokens/s, and time-to-first-token climbed from 0.46s to 32s — total bandwidth is near-fixed, so each added agent just thins everyone's slice.
- A community member's call was dead on: 1→2 agents lands near 1.5x, not 2x, because decode is memory-bandwidth-bound. Measured 1.57x decode-heavy and 1.51x prefill-heavy. The author's own prefill hypothesis missed, and they kept that miss in the write-up.
- Longer prompts batch better: sweeping prompt length from 170 to 3,100 tokens, the 2-agent gain climbed monotonically 1.52x→1.73x; prefill is compute-bound and parallelizes, decode is not.
- A dense 27B is the worst case: it re-reads every weight per token; an MoE activating 3B params per token has more headroom on the same bus.
The full run matrix is on disk and reproducible.
More from Infra
- Data Center Developers Offer $4M for Virginia Homes to Build Campus — Polymarket · 2026-08-20
- Seeking Best Open Source Harness for Qwen3.8:27B on macOS — emax · 2026-08-20
- Optimizing AI Costs: Right-Sizing Intelligence Spend with Model Mixtures — iamrobotbear · 2026-08-20
- AWS AgentCore adds runtime domain and date filters for Web Search — AWS ML Blog · 2026-08-20
- AWS automates mortgage document processing, cutting time by 70% — AWS ML Blog · 2026-08-20
- AWS patterns for async agent calls in serverless pipelines — AWS ML Blog · 2026-08-20