Cuddler compiler turns trained transformers into standalone executables, but alignment remains unsolved
Maui-The-Magificent · reddit · 2026-10-02
A systems engineer is building Cuddler, a model optimization re-representation compiler that converts trained transformer models into non-transformer representations that are their own executables. The architecture and runtime are nearly complete, and a converted qwen3.5:0.8b model already runs on a single core of a 7800x3D.
The remaining blockers are alignment: repetition loops on some prompts, complete gibberish on others, and no generic algorithm yet for aligning the internal topology during compilation — a problem the author suspects resembles quantization-time alignment. They're soliciting community ideas on how practitioners usually solve this.
More from Infra
- AI Capex Now Exceeds the Railroad Boom's GDP Share, Yet Demand Lags — FinanceYF5 · 2026-10-02
- Real-time AI serving costs up to 56x more than needed — one engine hits 56 sessions per H100 — Ok_boss_labrunz · 2026-10-02
- Cloudflare launches SQL API to query Workers logs and traces, replacing GraphQL plans — irvinebroque · 2026-10-02
- Cloudflare launches Web Search API via AI Gateway with Exa, Linkup and Ceramic — michellechen · 2026-10-02
- Lightpanda 1.0 ships: a Zig-built browser for AI agents, out of beta — jedisct1 · 2026-10-02
- Qwen3.8-27B coder quant fits a 24GB GPU with 262k context at 40 t/s — W61k3r · 2026-10-02