IOTA Decide 4B open-weight decision model hits ~11 decisions/sec on a Mac mini
const_reborn · x · 2026-11-05
MacrocosmosAI released IOTA Decide 4B v0.4, an open-weight (Apache-2.0) decision model: feed it a state (ticket, policy, agent trace) plus questions and one forward pass returns probabilities for every answer. Third-party benchmarks on a 64GB M5 Pro Mac mini in full bf16: 12.8s load, 10.7GB peak memory, yes/no in 88ms, 11 decisions/sec on short tickets, the bundled refund-policy example answered all 3 questions in 277ms with stable outputs (2.2% eligible, 86% refunds team, 63% low urgency). Longer states: 0.6s at 1k tokens, 2.6-3.0s at 4k. Numbers are a floor since linear-attention layers still run on unoptimized PyTorch paths. Trained across globally distributed GPUs via their SDK.
More from coding & agent
- Hugging Face ships safeagents scrub/publish to redact shared agent trajectories — yb2698 · 2026-11-06
- Hugging Face open-sources safeagents, a local proxy that blocks reward-hacking tool calls — yb2698 · 2026-11-06
- Synthesizing RL Envs from Open Trajectories: LFM2.5-24B Self-Scores Task Difficulty — yb2698 · 2026-11-06
- Hugging Face livestreams open RL training: trajectory-built envs, safeagents, LoRA evals — yb2698 · 2026-11-06
- Hugging Face synthesizes RL envs from open trajectories: Repair, Build, Extend, QA — yb2698 · 2026-11-06
- A 7-step blueprint for building AI agents, with a 10-framework comparison chart — mdancho84 · 2026-11-06