MLX-Serve 26.11.1 rewrite: plugin architecture adds models and quants without recompiling
TheMoonMidas · x · 2026-10-29
MLX-Serve 26.11.1 is a full engine rewrite built around building blocks and plugins. Anyone can write a plugin in C/C++/Rust/Zig etc. to add new models, quant types, media engines, or faster alternatives without recompiling the engine. Launch plugins include TurboEXL3 and Deepseek Sushi STQ; there is no central registry — plugins must be downloaded and compiled from GitHub (the author suggests having an agent review the code for security). The accompanying TurboEXL3 article argues EXL3 should be the default format for MoE expert weights on Apple silicon, challenging the current MLX affine 4-bit / MXFP4 / NVFP4 defaults.
More from Infra
- PGlite crosses 30 million weekly downloads as in-browser Postgres momentum builds — matei_zaharia · 2026-10-29
- Prime Inference scales from zero to 120B tokens per day on OpenRouter in one month — xeophon · 2026-10-29
- SJTU's TAO Cuts LLM Agent Serving Latency Up to 26.8% With Tool-Aware Joint Scheduling — AIOR-Research · 2026-10-29
- Nscale's unproven CEO chases $30 billion neocloud IPO amid AI turmoil — bloomberg · 2026-10-29
- Qwen3.8 Flash Next at IQ3_S hits 275 tok/s on a single RTX 5080 via Basalt fork — yehyakar · 2026-10-29
- Google Cloud CEO: Apple Scaling Foundation Model Pretraining on Latest TPUs — shayan_ · 2026-10-29