Tinkerbox runs LiquidAI d1-3B on Apple's Neural Engine at 40ms, 2.3x GPU efficiency
WeGoToMars7 · reddit · 2026-10-20
Developer WeGoToMars7 released Tinkerbox, an open-source project that runs small decision models efficiently on Apple's Neural Engine and hooks into OS automations via Shortcuts — no code needed to act on incoming email/iMessages, file changes, timers and more.
- Performance: LiquidAI d1-3B hits 40ms latency on warm short-context (<256 token) decisions on an M5 Pro; FP16 shows 100% top-choice agreement with FP32, <1pp probability drift; ANE delivers 2.3x decisions per joule vs GPU
- Why ANE fits: decision models skip token-by-token autoregressive decoding, avoiding the ANE's low memory bandwidth and static buffer limits; a tree structure evaluates multiple decisions in a single pass
- Details: native text+vision input, 32k context, M4/M5/M6 supported, macOS 27 required; AOT-compiled models and a SwiftUI app are available
- Roadmap: integrating vLLM d3 models based on Qwen3.5 (0.8B/2B/4B soon), plus embeddings, TTS, STT and image segmentation, aiming for a real ANE-optimized inference framework
Source, models and app are on GitHub/Hugging Face; the author asks for test results on other Macs.
More from coding & agent
- Claude Code v2.1.304 adds guided auto-mode setup, stricter permissions and usage warnings — ashwin-ant · 2026-10-20
- Pixel: a library bringing 120 FPS graphical UIs with Chromium WebView to the terminal — aidenybai · 2026-10-20
- Swarms argues Collective Superintelligence (CSI) will surpass AGI and ASI — KyeGomezB · 2026-10-20
- AI is already doing real work in RF chip design, FPGA optimization and 3D CAD, engineers say — mark_k · 2026-10-20
- Code review isn't dead: half of large AI-generated diffs add requirements you don't have — rseroter · 2026-10-20
- Rule of thumb: spend 90% of your RL task compute on trying to break them — marktenenholtz · 2026-10-20