Needle 2: A 14MB Agentic LLM Hitting 500 Tokens/sec on Raspberry Pi 5
Henrie_the_dreamer · reddit · 2026-08-11
Cactus released Needle 2, an ultra-compact agentic LLM designed for phones, wearables, micro-robots, and smart home devices. The model has 45M parameters (2-bit compression), runs as a single 14MB binary, and requires only 28MB of RAM.
Key Performance & Features:
- High-Speed Decoding: Achieves 500 tokens/sec on Raspberry Pi 5, 400-1,500 tokens/sec on VR devices like Meta Quest 3S, and 300-700 tokens/sec on sub-$200 phones.
- Ultra-Low Power: Significantly reduces MFLOPs per token compared to traditional transformers of the same width, fitting perfectly into the power budget of always-on devices.
- Focused Capabilities: Drops world knowledge and open-ended prose to focus purely on mapping natural language to structured function calls and data extraction.
It supports rapid fine-tuning on Mac/PC with minimal samples and includes a learned confidence score to escalate to cloud models when needed.
More from coding & agent
- How Do You Test AI Agents Before Production? Devs Discuss Hallucinations & Edge Cases — Saurabh4266 · 2026-08-11
- Developer Shares Workflow of Building an App Without Reading a Single Line of AI-Generated Code — tawnniee · 2026-08-11
- Browser Use CLI 3.0 Integrated into Hermes, Slashing Token Spend by 60% — Teknium · 2026-08-11
- Hermes Agent Launches Browser Automation Toolset with Multi-Backend Support — NousResearch · 2026-08-11
- Open Claw Agent Runs Overnight to Secure Fully Booked Health Club Spot — gregmushen · 2026-08-11
- Weights & Biases launches agent tracing that follows sessions, turns, and tool calls — wandb · 2026-08-11