Dev open-sources self-hosted model registry so inference workers boot when HF is down
cmm324 · reddit · 2026-11-19
The author built Model Delivery for Otium, a batch inference system that spins up workers on AWS Spot instances. A self-hosted registry mirrors pinned Hugging Face revisions ahead of time, preserving revision IDs and verifying file hashes, so workers can boot even if the Hub is down, a repo is removed, or upstream access is cut during a security incident.
- Point HFENDPOINT at the registry and vLLM downloads weights as usual.
- Ships as a single binary with SQLite and files on disk.
- Demo shows byte-for-byte identical downloads of a five-file model via the HF CLI after shutting down the local hub.
More from Infra
- Oracle co-CEO admits 'classic mistake' selling data centers to towns — jeremyakahn · 2026-11-19
- Surface Laptop Ultra With NVIDIA RTX Spark Runs 94GB Model, Finishes 4+ Enterprise Agent Workflows Per Hour — ryanshrout · 2026-11-19
- Nace ships 71.8GB DeepSeek V4.1 Flash coding quant with 39% lower KL than rivals — m4rkGS · 2026-11-19
- Architect Labs unveils Frostbite, an agent-co-designed CPU claiming 4.6x perf per watt over Arm — IgorCarron · 2026-11-19
- Nvidia's unusual delay in filing its 10-Q sparks Wall Street intrigue — analyst explains why — TechCrunch AI · 2026-11-19
- Oxmiq Offers AI Chip IP and Tools to Indian Teams, Discusses Custom Silicon Economics — RajaXg · 2026-11-19