Rubick: an Open-Source Gateway Serving Chat, Embeddings and Calibrated Decisions from One Model
wFXx · reddit · 2026-10-19
Inspired by IBM's Granite Switch, the author open-sourced rubick, a proxy gateway that points at any llama-served endpoint and exposes regular LLM chat, jev-like systemone decision, and embeddings endpoints from the same model. It reuses kv-cache, requires no model swapping, supports plug-and-play aLoRAs, and ships a 1-hour end-user-hardware aLoRA training recipe, benchmarks, use cases and Docker setup under Apache 2.0. Built on numind's vLLM PR; the author hopes llama.cpp and other engines adopt the ideas.
More from coding & agent
- Anthropic Built Scion, Its Own Source Control System, as Merged Changes Grew 12x to 30,000/Day — Pavan_Belagatti · 2026-10-19
- No editing software: one creator made a full AI video just by chatting with Claude — aziz4ai · 2026-10-19
- 20+ exposed panels reveal open-source AI hacking agent Cairn in real cyberattacks — Hesamation · 2026-10-19
- The Vibe Coding Curse: Understanding AI Code Defeats the Speed, Not Trusting It Blocks Shipping — MoonL88537 · 2026-10-19
- Six months of agent-memory provenance fields nobody could see: a Reddit stranger found the gap — Neither-Witness-6010 · 2026-10-19
- The AI coding curse: skipping code understanding makes you helplessly dependent on the model — MoonL88537 · 2026-10-19