Prefill beats decode: SGLang wins agent runs on DGX Spark despite slower tok/s
AIFlow_ML · x · 2026-11-20
- redp314 benchmarked the same Qwen3.8-27B coding-agent workload on a single DGX Spark, comparing TensorFold vs SGLang.
- TensorFold generated faster (52.7 vs 40.8 tok/s), but SGLang read prompts 60% faster (prefill) and won the overall agent run.
- Takeaway: agents read far more tokens than they write — output tok/s is a misleading metric for inference engines in agent workloads; benchmark prefill instead.
More from coding & agent
- Open-source repo offers 500+ AI-generated low-poly 3D landmarks for maps under CC0 — tomjohndesign · 2026-11-20
- SwirlAI: nine places to plug decision models into agentic RAG, with runnable code — blaizedsouza · 2026-11-20
- Agent safety rule: human-approve any irreversible, externally visible action — and show the exact action, not 'proceed?' — Beneficial_Use2116 · 2026-11-20
- Viral 'Claude Code Harness' article argues the harness, not the model, makes the agent — blaizedsouza · 2026-11-20
- Running ComfyUI fully via Claude in a container — no manual UI anymore — xWranglerWranglerx · 2026-11-20
- AI writes all code to make a real robotic arm play chess, open-sourced — FinanceYF5 · 2026-11-20