GLiNER2.5-Decide hits 7.6ms local routing calls on M5 Pro, an LLM-free alternative to typed JSON
ivan_digital · reddit · 2026-09-27
The author, maintainer of speech-swift, ported Fastino's GLiNER2.5-Decide to MLX Swift and compared it with TypeSafe's Jev for label-probability classification.
- Fastino's headline 60.2% vs 57.6% benchmark was against JevK5, an open reproduction on a 4B-class Qwen3.5 decoder — not the commercial Jev API.
- Local benchmark (M5 Pro, INT8): 7.6ms per routing call, 0.85GB memory; all three precisions match PyTorch outputs on 24 reference cases.
- Trade-offs: Jev handles up to 64k tokens with rubric scores and yes/no probabilities; the Swift port is 512-token, single-label classification plus entity spans, fully offline.
A lighter, lower-latency alternative to LLM structured output for routing tasks.
More from coding & agent
- DHH: Agent-led development demands a business-owner mindset focused on outcomes — alexmacgregor__ · 2026-09-27
- App Screenshot Skill Hits 7,000 GitHub Stars, Adds 6 New Styles, Free & Open Source — moeinteractive · 2026-09-27
- IBM open-sources Docling, a free Python library that converts any document to data — mdancho84 · 2026-09-27
- Dev reflects: coding now feels like a waste of time when LLMs solve 99% of problems — justalexoki · 2026-09-27
- Reward hacking bugs revealed: unpruned git history let models peek at patches, edit tests in shared sandbox — willcb · 2026-09-27
- A Content Pinball Machine built entirely with Claude Opus 5.5 satirizes viral randomness — CurieuxExplorer · 2026-09-27