Zero-token chatbot: picks from 400 pre-written replies, ~100x cheaper
dwarfLevi · reddit · 2026-09-20
- The author built a conversational AI that generates no tokens: TypeSafe's Jev (a non-generative, probabilistic-judgment model) classifies the user message (18 parallel questions in one 400ms call via speculative fan-out), code fetches 10 candidates from a bank of 400 hand-written replies, and a second 400ms call scores each on 6 dimensions with context-aware weights.
- Falls back to Claude Haiku only when scores/confidence are low. End-to-end 800ms at $0.0001/message — roughly 100x cheaper than generative output.
- Benefits: zero hallucination (all text pre-written), full transparency into intent and scores, and surprisingly strong performance on open-ended philosophical questions where picking the right framing matters more than generating content.
More from coding & agent
- Devin's native video capture of test runs shows its cloud-first edge — brandon_galang · 2026-09-21
- Feeding task descriptions to JEV yields workflows across hundreds of apps in seconds — gaganghotra_ · 2026-09-21
- 'Agent Experience' emerges as a new UX theme: some of your users are agents now — YvesMulkers · 2026-09-21
- Anacondahood launches: conda research + Robinhood MCP + onchain rails for AI trading agents — teoliphant · 2026-09-20
- Camera-only autonomous drone clears obstacle course with Jev judgment model, built in 15 minutes for $0.10 — alex_verem · 2026-09-20
- Spotify Turns Internal Coding-Tool Chaos Into Xirp, a Mac App for 50+ Parallel AI Sessions — shashib · 2026-09-20