hunch Turns EmbeddingGemma 2 Into a Real-Time Local Semantic Decision System: 22ms on CPU, Under 3 Cents Per Million Tokens
aigclink · x · 2026-11-05
The open-source hunch project builds a real-time local semantic decision system on Google's EmbeddingGemma 2 that learns continuously on CPU.
- 22ms per text decision, 100ms for images, under 3 cents per million tokens, fully local
- Architecture: EmbeddingGemma 2 + lightweight classifier + teacher-model distillation — an LLM acts as teacher while a small local model gradually takes over high-frequency judgments
- Use cases: tool-call failure detection, log anomaly checks, intent classification, agent task routing
- Complex queries still go to the LLM, forming a continuously learning, cost-decreasing loop
Related event: Open-Source hunch Delivers Millisecond Local Decisions on EmbeddingGemma 2(3 posts)→
More from coding & agent
- If an agent misuses my valid credentials, what can tell it wasn't me? — Thessaly-Sorties · 2026-11-05
- FlutterFlow launches AI agent to build native mobile apps from a single prompt — massimosgrelli · 2026-11-05
- Hugging Face launches ML Intern, an open-source agent that does ML for you — Gradio · 2026-11-05
- Alibaba open-sources qz, a browser agent that runs inside your signed-in Chrome — thisdudelikesAI · 2026-11-05
- Blackstone and Norm AI CTOs: agent capability made our workflow code obsolete — AI Engineer · 2026-11-05
- Rentcast MCP Server open-sourced for property data, valuations and rental estimates — modelcontextprotocol · 2026-11-05