Melbourne lab i14labs launches Sugar Glider: O(1) inference architectures tested up to ~1T params
kimmonismus · x · 2026-11-10
Melbourne lab i14labs launched Sugar Glider, a family of LLM architectures claiming constant-memory inference for existing open-weight frontier models convertible in hours:
- O(1) per-token inference with a fixed-size KV cache — memory and compute per token stay flat as context grows
- Claims frontier long-context retrieval quality, tested on models up to 1T parameters
- 1B-token context retrieval: 100% single-needle, 90.5% multikey-1
The team credits decades of work on human memory and language models. The author argues that with power, grid and datacenter buildout hitting limits, squeezing more from existing hardware is becoming a necessity. Waitlist is open for early access.
More from Infra
- Shopify's 'Flywheel': a continual learning loop on PyTorch + vLLM that beats frontier models at a fraction of the cost — PyTorch · 2026-11-10
- AFMGUI: a clean one-command chat app for Apple's on-device Foundation Models — BLCNYY · 2026-11-10
- AI hardware design gets faster, manufacturing matters more: owning the full stack is the moat — espricewright · 2026-11-10
- Anthropic went from GPU-constrained to available everywhere, says dev on Fable 5.2 rollout — ivan_bezdomny · 2026-11-10
- NVIDIA's 120B MoE Nemotron 3.5 Super VL VLM arrives on SageMaker JumpStart — AWS ML Blog · 2026-11-10
- DeepSeek is 44x cheaper than Claude Fable 5.2 per task: $0.60 vs $26.54 — Yuchenj_UW · 2026-11-10