Artificial Analysis open-sources agent inference benchmark, first results for DGX Spark, RTX 5090, M5 Pro
ArtificialAnlys · x · 2026-09-30
Artificial Analysis open-sourced AA-AgentPerf-Local, a tool that replays real agent trajectories (8 tasks, 168 turns, 56K-token contexts) against OpenAI-compatible servers to benchmark local agentic inference. Initial results cover NVIDIA DGX Spark 128GB, RTX 5090, AMD Ryzen AI Halo, and MacBook Pro M5 Pro, with Qwen3.5/3.6/3.8 and Ling 3.0 Flash at 4-bit quantization across CUDA, ROCm, Vulkan, and Metal.
Related event: Artificial Analysis Open-Sources Local AI Agent Benchmark(2 posts)→
More from coding & agent
- shadcn boosts @uisdk: one-shot UI rebuild that's instantly E2E-testable by AI — shadcn · 2026-11-16
- AI agent deletes 43,599 lines of OpenClaw test code in one PR — vincent_koc · 2026-11-16
- JPMorgan shows Jarvis, its internal agent assembly line, at Interrupt NYC — LangChain · 2026-11-16
- Dev clarifies: Apple doesn't ban feature flags, only fooling App Review — talkaboutdesign · 2026-11-16
- Antigravity Matches Claude Cowork in Tests, While Gemini Tasks' Purpose Remains Unclear — sebajun2 · 2026-11-16
- One 16GB GPU runs songs, voice, images, 720p video and local RAG — all offline and open source — llama-of-death · 2026-11-16