Running an 8-agent research swarm locally on a 48GB M4 Pro with one 40B Flint model
cong_ml · x · 2026-10-27
- A thread shows how to run a full research swarm on a Mac: eight agents share a single local 40B Flint model in llama.cpp on a 48GB M4 Pro, writing code, testing ideas, and building on what works.
- Key trick: batching. Eight concurrent requests yield 1.9× the aggregate tokens/sec of one request at a time (averaged over three tasks via Metal). Each agent writes slower, but together they produce nearly twice as much.
- Setup: give the agents a problem and a score to beat — everything, including the model, stays on-device with no cloud APIs.
More from coding & agent
- "Error correction in a loop with LLMs is underrated" — burny_tech · 2026-10-28
- Silicon Valley wrote 70+ agent infra reports in 18 months; Chinese VCs barely noticed — FinanceYF5 · 2026-10-28
- Vibecoded Factorio-SimCity hybrid lets you build a utopian city in the woods, works on mobile — AndyMasley · 2026-10-28
- Nitro launches: formally verified agent compiles SQL to Rust for 10x faster, 3x cheaper ETL — kuza55 · 2026-10-28
- Dev builds a Grok bot connector to ping Apple devices via Find My — Baconbrix · 2026-10-28
- What evidence do you actually need to debug an AI coding session? — Additional-Ladder-24 · 2026-10-28