ASRN architecture copies long-range context with 69ms at 256k tokens, beating Mamba-2 and RWKV-7
Mean-Disaster8380 · reddit · 2026-10-25
A Reddit post introduces ASRN (Adaptive Sparse Recurrence Network), a language-model layer achieving linear-memory long-range exact copying:
- Each token attends only to its C most recent neighbors in learned sign-hash buckets and copies the token following each match, enabling exact retrieval of earlier spans
- A learned proper-time clock Bt sets the forgetting rate, with Hawkes-style self-excitation and a repetition-streak gate deciding when to copy
- Benchmark chart on one A100: at 256k tokens ASRN 1.0 takes 69ms round-trip vs 174ms for a Mamba-2 block, 519ms for RWKV-7, and 1486ms for FlashAttention-2; attention remains faster below 18k tokens
- Experiments are small scale (≤50M params); preprint DOI and GitHub code are available
More from Research
- ExCAM: explainable metric spots LLM cultural errors at 80% accuracy — prajdabre · 2026-10-25
- Three EMNLP 2026 papers: vector merging beats data mixing — prajdabre · 2026-10-25
- MIT-led team builds AI engine to match diagnostic devices to Long COVID needs across US counties — ahandvanish · 2026-10-25
- AI drug-discovery skeptics have a point: binding simulation accuracy is just 49%, data costs 30x longer — citrini · 2026-10-25
- Open-source DNA model Carbon-X (14.8B) nears frontier scores on genomics at 1/1800th the cost — aigclink · 2026-10-25
- MemT 1.0: 293B memory-native MoE puts persistent memory inside the forward pass — AKAMA199 · 2026-10-25