Help: Docker Deployment of Local Qwen3.8 FP8 Failing
EbbNorth7735 · reddit · 2026-08-24
A user is trying to run a locally stored Qwen3.8 27B FP8 model within a Docker container but facing path configuration issues. vLLM fails to recognize the mounted local path as a model source, defaulting to a Hugging Face reference instead. They are requesting a working Dockerfile example to correctly load the local model.
More from Infra
- Context Layer: A User-Controlled Contract for Cross-Model/Agent Context Transfer — sierracatalina · 2026-08-24
- OpenRouter: token demand will far outpace compute; heterogeneous hardware including retail Macs is key — gajesh · 2026-08-24
- Decentralized Inference on MacBooks: Earn $200-$500/Month via darkbloom — siddharthnibjiya · 2026-08-24
- Google Colab adds SSH and CLI support for seamless cloud workflows — HankYeomans · 2026-08-24
- Speculative Decoding: Obsolete by the Time You Implement It — dejavucoder · 2026-08-24
- EigenLabs Serves 5.23B Tokens in a Day, Hits $134K ARR — AccBalanced · 2026-08-24