Infinity: a Rust agent runtime that fits 75k agents in a Raspberry Pi's RAM

ShadajL · x · 2026-10-24

Shadaj L unveiled Infinity at AGNTCon, a Rust framework for massively concurrent agentic systems—"what async did for threads, for agents."

How it works

Agents spend most of their lives waiting (inference, tool calls, humans); Infinity suspends idle agents by default (scale to zero) and wakes them on the next event
An idle agent is pure data: after 20 tool-calling turns it occupies 103 KB, so 75k agents fit in 8 GB RAM (slides show 80k in 8.4 GB)
Async primitives: tool calls stay open as streaming subscriptions, spawnthread forks parallel subagents, sleep awaits timers/events without polling
Actor-style architecture: agents share nothing and communicate via ordered mailbox messages

Big cost win for workloads with many long-horizon, low-frequency agents (builds, webhooks, human-in-the-loop).

More from coding & agent