Mica v0.1 4B: Open Decision Model Runs on 8GB GPU, Trained for Under $30
Top-Evidence174 · reddit · 2026-09-26
The author released Mica v0.1 4B (Apache-2.0), a small decision model for agent loops — gates, routers, and "should I ask the user or just act" checks.
What it does: Given a state, a question and allowed answers, it returns a calibrated probability per answer: yes/no, a choice among 2–255 options, or a score with 2–10 levels. It never generates text — one prefill, then reads the logits of option labels at the answer position. It speaks the TypeSafe /v1/systemone format.
How it's built: Qwen3.5-4B with a rank-16 LoRA on all 32 layers (attention and Gated DeltaNet), merged; no new heads. 34k source decisions expanded to 77,732 training rows (34.7M tokens), roughly half English and half Korean, across 12 areas. Plain cross-entropy, one epoch, one global temperature. All experiments plus the final run cost under $30 of rented RTX 3090 time.
Results (7,328 held-out decisions, English subset): Jev 1.13 74.1, Mica 4B 67.0, JevK5 4B 61.0, Kev 4B 57.0, Qwen3.5-4B base 55.0. On public sets Mica scores 94.4 on SemIf, 69.5 on JevBench hard (64.9 via the official runner), and 53.0 on MMLU-Pro vs Jev's 82.3.
Where it's useful: Resists in-data prompt injection — with a note telling the judge to pick a wrong option it still gets 69% right (81% without), while Jev drops to 18% and Kev to 31%. Well calibrated: when it says ≥0.9 it's wrong only 2.5% of the time (ECE 5.4%). Q5KM is 3.5GB with no measurable accuracy loss. On an RTX 3090 it runs at 47ms median (Q4KM), faster than Kev 4B (76ms) and JevK5 (99ms).
Limitations: Weak on knowledge-heavy questions; long English policy documents are its weakest set; notes inside the state still nudge it; it doesn't distinguish reversible from irreversible actions well; less confident on harder reasoning items. Weights and code are open, with a one-line Docker command.
More from coding & agent
- Hamel Husain and guests livestream building a tier list of AI engineering and evals techniques — HamelHusain · 2026-09-26
- dhh: Human-Only Code Review Will Soon Be Seen as Deeply Irresponsible — deliprao · 2026-09-26
- Grok bot packs 750GB of open-source AI models into one downloadable archive — RachelVT42 · 2026-09-26
- Codex is down, so this researcher built an AI chain to watch it come back — Yuchenj_UW · 2026-09-26
- Claude Opus 5.5 one-shots a promo video in Cursor: has video animation been solved? — JosephJacks_ · 2026-09-26
- dhh: Opus 5.5 one-shot ported Rust screensaver engine to x86-64 asm, 17x faster — zack_overflow · 2026-09-26