llama-doctor: CLI that diagnoses your llama.cpp server from /metrics with severity-graded findings
msalsas · reddit · 2026-10-19
Reddit user msalsas open-sourced llama-doctor, a diagnostic CLI and live dashboard for running llama.cpp or Unsloth Studio servers.
- Points at a server, pulls /metrics, /props and /slots, and reports severity-graded findings: prefill vs decode, queue pressure, prefix cache reuse, memory headroom
- Exit codes map to severity so cron jobs can gate on results
- Read-only, stdlib-only, Python 3.10+
Code is on GitHub; the author welcomes PRs, especially crash reports from unusual server setups.
More from Infra
- Qwen3.5-0.8B 4-bit spotted running on Rabbit R1 via wllama — ngxson · 2026-10-19
- ORL trains diffusion LMs on their own trajectories, boosting a 4B model's throughput by 6.9x past speculative decoding — burny_tech · 2026-10-19
- 4 free CUDA resources to learn GPU programming from scratch, from NVIDIA to Oxford — techNmak · 2026-10-19
- Running Moss image model on a 4GB RTX 3050, optimized down to 62s per image — sanketshinde2004 · 2026-10-19
- Barclays: pure-play data center bond spreads hit ~209bps as market prices in contagion — SumitGup · 2026-10-19
- GPU programmer says the field needs lighter, more predictable inference engines — brandon_xyzw · 2026-10-19