NuMind open-sources vLLM plugin: free multimodal embeddings from your already-loaded LLM
Gailenstorm · reddit · 2026-10-12
NuMind (maker of NuExtract3) released an Apache-2.0 vLLM plugin that extracts text/image embeddings directly from an LLM already being served by vLLM—no separate embedding model deployment needed.
The idea: to predict the first output token, the model already computes internal hidden states of the prompt; the plugin exposes the last hidden state via the chat API (OpenAI-client compatible, opt-in on one-token non-streaming requests). It builds on the E5-V paper, using prompts like "summarize the above image in one word" to concentrate relevant information at that position.
Use cases include local RAG, semantic search, document/image clustering, and training small classifiers. The author notes a hidden state isn't automatically a good embedding—quality depends on model, prompt, and task. An upstream vLLM PR (#57185) is open, and a llama.cpp port may follow.
More from Infra
- UK Minister Backs Hardware-Enabled AI Verification, Betting on Britain's Chip Design Expertise — S_OhEigeartaigh · 2026-10-12
- GPU rentals up 27% in six months while fixed-intelligence prices fell 63% — jon_durbin · 2026-10-12
- Broadcom's Hock Tan is betting the company on Anthropic and OpenAI succeeding — firstadopter · 2026-10-12
- AI providers processed 39 quadrillion inference tokens in September, up 7.9x YoY — SuB8u · 2026-10-12
- Cheap drones could torch a $50B data center, researcher urges anti-drone defense — Dr_Singularity · 2026-10-12
- OpenEnv Arena offers free GPU compute to train your environment, weights open sourced — ben_burtenshaw · 2026-10-12