IBM/Cornell's LPT shares KV-cache across looped transformer passes, cutting memory 79% while improving quality

burny_tech · x · 2026-10-30

IBM Research and Cornell introduce the Looped Prediction Transformer: only the first loop writes a KV-cache; later loops read it plus a short local window.

• Surprising finding: sharing memory improves rather than hurts quality — shared and local memory learn different representations, later loops attend mostly to the shared cache, which also acts as a gradient highway
• At 150M-1B params, the hybrid variant lowers FineWeb-Edu validation perplexity by 1.12-1.82 vs a same-size standard Transformer while using 76-79% less context memory
• At 1B, the hybrid achieves 10% lower perplexity than a five-loop standard Transformer

The work sets a new quality-memory frontier for looped models that trade repeated layers for parameters.

Original post →

More from Infra

• Amazon guides AWS to 40%+ growth through 2028, eyes $1T business in 5-10 years — mitchdeg · 2026-10-30
• By 2028, 'country of geniuses in a datacenter' will be limited only by datacenter count — timshi_ai · 2026-10-30
• 797GB of CPU-side n-gram lookups match 57% more compute on 1B LM training — PangWeiKoh · 2026-10-30
• Redditor jokes about finding racks of idle GPUs at a local data center — Beneficial-Ad-8127 · 2026-10-30
• Intel shares plunge 20% as AI-driven memory shortage hurts its PC chip business — pstAsiatech · 2026-10-30
• Wafer hits 2 trillion daily tokens on OpenRouter, doubling from 1T in just 4 days — ycombinator · 2026-10-30

Infra channel →