Attention at Absolute Zero: cold-read attention layers make reasoning exact at 100x scale

Mostafa 'Neo' Mohsenvand · hf · 2026-11-19

The paper "Attention at Absolute Zero" builds a higher-order attention layer whose state lives on pairs of items, with each third item judged as a "witness," read at zero temperature: keep the best witness and its ties, pass on only bits comparing counts, and rebuild state from the input each loop.

Original post →

More from Research

Research channel →