WarpState: Two-Timescale Memory to Replace KV Cache

An experimental 2B-parameter architecture called WarpState uses two-timescale compact recurrent memory to replace ever-growing KV cache, applying exact causal attention within 128-token blocks.

2026-09-20 ~ 2026-09-21 · 2 related posts

• WarpState: a 2B-param architecture with dual-timescale associative memory replacing ever-growing KV caches — zemondza · 2026-09-20
• WarpState: 2B-param architecture swaps growing KV cache for fixed-size dual associative memory — zemondza · 2026-09-21