Latent Lattice Language Models: adding a horizontal computation axis to the residual stream

HanGuo97 · x · 2026-11-20

Shivam Duggal proposes Latent Lattice Language Models (LLLMs): instead of processing each token only through a single vertical residual stream (reading/writing memory and applying MLPs layer by layer), the approach augments it with a horizontal axis of computation—a lattice of latents that interact with each other.

Framed as a handshake between RNNs and Transformers, the idea has drawn interest from researchers who note that how latents should flow and interact remains largely unexplored—an architecture-level direction worth watching.

Related event: MIT and Adobe Propose Latent Lattice Language Models(3 posts)→

Original post →

More from Research

Research channel →