5ms Router Slashes LLM Costs by 96%
Negaaaa7 · reddit · 2026-07-19
To cut LLM API costs, the author built a 0.57 MB router that decides between local and cloud models, boasting an inference time of roughly 5 ms without needing embedding models or vector databases.
The router simultaneously handles three tasks:
- Local/cloud routing
- PII detection
- Jailbreak / injection identification
They share their test results, including an offline evaluation of 2,022 real prompts, jailbreak detection rates, false positive rates, and known failure modes (e.g., certain "act as..." role prompts easily trigger false positives).
It's currently available in three formats: a VS Code/Cursor extension, an npm SDK, and a hosted API. The author emphasizes that the extension is merely a frontend, while the actual local classifier runs within the SDK.
Related event: Tiny 0.57MB Router Slashes LLM Costs(2 posts)→
More from coding & agent
- Automatic end-to-end testing now reaches the backend in one integrated flow — blixt · 2026-07-22
- Developers are using Lovable and Google AI Studio for fast UI refreshes — rseroter · 2026-07-22
- Turning 3D Gaussian Splatting into a Browser Experience: A Practical Guide — willeastcott · 2026-07-22
- Claude Code creator says model capability is outrunning the product surface — haider1 · 2026-07-22
- Omar Sarar says agent workflows now need a custom orchestrator — omarsar0 · 2026-07-22
- AgentVille runs local Ollama agents in an open-source pixel town — GeraAI_WW · 2026-07-22