jevcache memoizes model decisions by (model, schema, state) — repeat calls cost $0 and return in ~0ms
JiliJeanlouis · x · 2026-09-20
A new open-source tool called jevcache memoizes model decisions keyed by (model, schema, state), so the same decision never runs — or bills — twice.
Key claims:
- Cost: 60–80% of an agent's decisions repeat (loops, retries, idempotent tools); cache hits cost $0. Its demo shows 784K of 1M calls served from cache.
- Determinism: same input yields the same answer, replayable in CI or offline without model-version drift.
- Shareable: caches can be published so teams recall against shared decisions via a file or hosted index.
- Latency: local hits return in-process at 0ms with no round-trip.
It ships as a 2MB binary installed via curl -fsSL jevcache.sh/install | sh, runs locally with your own key and bill, and only a hash ever leaves the machine. Example: first call to jev-1 at 184ms for $0.000042, subsequent calls free from cache.
More from coding & agent
- YC partner: VM-based browser agents may make bare metal harnesses like OpenClaw obsolete — MurrLincoln · 2026-09-20
- Running Comfy workflows via a frontier model: one dev's local image-gen pipeline — mccoypauley · 2026-09-20
- AI Engineer talk: smart software decides, humans sit at approval gates — ccerrato147 · 2026-09-20
- TypeSafe ships Jev: typed calibrated decisions, no RLHF and no text generation — ccerrato147 · 2026-09-20
- Smart software is a flowchart where the nodes think, not a chatbot bolted on — ccerrato147 · 2026-09-20
- Why every LLM needs a human in the loop: we literally put them there via RLHF — ccerrato147 · 2026-09-20