swallm Brings Sliding Window Attention to HF LLM Inference

A developer released swallm, an open-source project that adds sliding window attention to HuggingFace causal LLMs at inference time without retraining, cutting KV cache to about 3.5MB for 32K–64K contexts.

2026-09-06 ~ 2026-09-06 · 2 related posts

• Dev Ships Open-Source Sliding-Window Attention for HF LLMs, Hits 3.5MB KV Cache at 32K Context — ahsaor8 · 2026-09-06
• Sliding-Window Attention at Inference Shrinks 64K-Context KV Cache to Just 3.5MB on Pretrained LLMs — ahsaor8 · 2026-09-06