Architecture papers reading list highlights Mamba-3 non-commutative state tracking advance
zmkzmkz · x · 2026-10-10
samcompute shared a reading list of architecture papers, highlighting a recent progression in what SSMs can track: parity -> modular arithmetic -> non-commutative state tracking.
Standout: a paper uploaded to arXiv just 2 weeks ago by a Japanese team shows Mamba-3 can perform length-generalizing non-commutative state tracking (S5) with simple adjustments to the A-matrix (NPLR). The list also includes looped transformers and other directions, curated with zmkzmkz, inventor of multi-layer key-value sharing.
More from Research
- SF event Oct 22 asks what it takes for AI to run scientific research end to end — nlarusstone · 2026-10-10
- Paper argues nested von Neumann architecture can make a million processors act as one computer — bronzeagepapi · 2026-10-10
- UPenn paper unifies diffusion and autoregression on one corruption lattice to predict decoding costs — upenn · 2026-10-10
- CMU's Keenan Crane wraps AI-assisted 3D modeling thread, releases all model files under CC0 — keenanisalive · 2026-10-10
- From Maxwell's field lines to Jim Blinn's blobby molecular visualization — keenanisalive · 2026-10-10
- OmniSeek turns Omni-LLMs into evidence-seeking agents, +15.5 points on VideoHolmes — mohitban47 · 2026-10-10