Taco: hardware-agnostic LLM compression with one hash across 7 backends
phoenix10701 · reddit · 2026-10-20
Open-source Taco tackles a core LLM compression problem: the decoder must compute exactly the same probabilities as the encoder, but float math differs across hardware — a float GPT-2 encoded on a Mac decoded 0 of 4 windows on x86. The approach: integer-only inference (int8 weights, fixed rounding, lookup tables), an exact float32→integer converter (mantissa × 2^exp), and conformance vectors, yielding one identical hash across 7 backends (Mac CPU/Metal, Linux CPU/CUDA, Windows CPU/CUDA/Vulkan).
More from Infra
- A Dell workstation with an RTX Pro 6K inside costs less than the card alone — Sentdex · 2026-10-20
- Liquid cooling leaks can take an entire AI factory down for days, expert warns on monitoring silos — ai · 2026-10-20
- China proposes 40 new tech export controls, including 23 AI technologies — pstAsiatech · 2026-10-20
- Stripe details scaling Radar: fuller training history plus agent-assisted ML cut fraud — emilygsands · 2026-10-20
- Stripe: agents now run the ML experiment loop, blocking tens of millions in fraud a year — emilygsands · 2026-10-20
- Google Cloud rethinks Kubernetes for agents: suspend/resume as core primitive — rseroter · 2026-10-20