A Minimal 1100-Line Proxy for Local LLM Servers with Per-User Keys and Rate Limits
Yulya_N8FAD85042 · reddit · 2026-07-30
A developer has open-sourced smol-llm-proxy, a lightweight proxy tool designed for local LLM servers. Built for scenarios where llama-server instances are shared among users, its core features include per-user API key management, token accounting, and RPM/TPM rate limiting.
Key highlights:
- Extremely Lightweight: Uses only SQLite without requiring Postgres or Redis. The codebase is strictly kept around 1100 lines via CI.
- Low Overhead: Proxy logic takes 0.13 ms per request with a minimal memory footprint of 53 MB.
- Focused Features: Supports model routing, aliases, and streaming without storing conversation content. The author notes the code was mostly written using local Qwen models via opencode.
More from coding & agent
- Deep Agents v0.7 Released: Slashes Default Context Tokens by 65% — hwchase17 · 2026-07-30
- Developer Builds Fully Procedural 3D RPG Game Using Claude — chrisfirst · 2026-07-30
- jasonkneen Launches Browser Agent with 3D Voxel Avatars — jasonkneen · 2026-07-30
- Open Source Sticky Notes Plugin for Hermes Desktop Manages Fleeting Ideas — Teknium · 2026-07-30
- Agensis Launches Shared Workspace for Humans and AI Agents — jasonkneen · 2026-07-30
- Machinen Desktop: A Spatial Desktop Environment Built for AI Agents — kieranklaassen · 2026-07-30