Open-source Mica v0.5: a 4B decision model at 19.8 ms per call, 25-340x faster than APIs
Top-Evidence174 · reddit · 2026-10-22
The author released Mica v0.5, an Apache-2.0 4B decision model that runs on a single consumer GPU.
Speed (RTX 6000 Ada):
- 19.8 ms per decision at 256 tokens, 1.2 s at 32k
- Up to 3.8x faster than other 4B judges, 12-19x faster than Perplexity's 27B decider, and 25-340x faster than API decision models (Solar Decide 35B at 6.74s is 340x slower)
- Mica Fast makes long inputs another 1.4-1.5x quicker; the INT8 build is 2-2.7x faster than the Q80 GGUF on llama.cpp
Accuracy: 84/111 on JevBench Hard, best of any 4B model tested (only Microsoft's 9B Decision-1 scores higher); 58.8 on JDI, ahead of every 9B and 35B tested. The author admits top JDI 4B models are benchmark-focused and trained on benchmark-style data; he used same-domain data (uncontaminated) but focused on real work, drawing on 200+ real use cases from Reddit and Twitter.
Robustness: 97.3% answer stability under option shuffling; supports abstention thresholds; yes/no, multiple choice, and 2-255 scoring with 32k-token inputs. BF16/Q80/IQ4XS GGUFs available; runs on 8 GB cards.
More from Models
- OpenAI's Decisions API hits GA: near real-time routing up to 10x faster than GPT-6 Luna — OpenAIDevs · 2026-10-23
- DeepLearning.AI weekly: GPT-6.1 hits 97.9% byte parity on COBOL migration — DeepLearningAI · 2026-10-23
- Marker's lite mode scores 75.1 vs 42.3 for Azure Read on OmniParseBench — VikParuchuri · 2026-10-23
- Datalab launches lite OCR at $0.50 per 1,000 pages, 20-30x cheaper than Azure, Google and AWS layout APIs — VikParuchuri · 2026-10-23
- LiveNerf's 29-day data finds no nerfing of Claude Opus 5.5 — TheOnlyVibemaster · 2026-10-23
- PHP Foundation reveals Anthropic's Claude cyber model scanned PHP, validated with OpenAI's Daybreak Red — saibharadwaj · 2026-10-23