HybridInfer: Router auto-falls back to cloud when local model wedges
simrankoulsm · reddit · 2026-09-02
HybridInfer is an open-source reliability-aware router for hybrid inference.
Core Features:
- Routes requests to local models (Ollama) first.
- Auto-switches to a remote cloud model if local stalls, OOMs, or errors.
- Learns from failures to proactively route problematic prompts.
- Manages model rotation with automatic health checks.
Acts as an OpenAI-compatible server, providing local-first inference with automatic cloud fallback.
More from coding & agent
- Claude-written test harness let 31KB fake archives land on a real backup server — Grimmoner · 2026-10-18
- Matt Pocock's pre-commit hook caps agent-written files at 5,000 tokens — mattpocockuk · 2026-10-18
- Backups silently failed for 10 days: three monitoring blind spots lined up — Grimmoner · 2026-10-18
- Author Laurence Moroney: write your ugly first draft yourself, then let an LLM edit it — lmoroney · 2026-10-18
- Open-weight TYGER speech decision model makes 80ms turn-taking calls for full-duplex voice agents — solyarisoftware · 2026-10-18
- Devs: Opus 5.5 on medium beats Opus on max — eyishazyer · 2026-10-18