Korean paper Naju splits retention and writing gates in a new state-space model
TheTuringPost · x · 2026-07-29
Naju separates retention and writing in a state-space model
A Korean paper introduces Naju, a native discrete state-space model that brings back the LSTM idea of separate gates for retention and writing.
- One gate controls how much of the existing state is kept.
- Another gate controls how strongly new information is written.
- Unlike coupled single-gate designs, Naju allows both preserving and adding information at the same time.
The model was trained on sequences of 512 tokens and tested at 2,048 tokens, where it reached 0.99 retention accuracy and 0.89 overwrite accuracy. In the paper’s comparisons, xLSTM did better at remembering, while GLA did better at overwriting, but neither matched both behaviors equally well.
The authors also note two limits:
- Speed: at 32K tokens, Naju was 2.2× slower than Mamba and 3.3× slower than Mamba-2.
- Scaling: the evaluation is still small-scale, using WikiText-103 with a 1.2B-token training budget, so large-scale behavior remains unclear.
Related event: Korean Team Introduces Naju Model with Dual-Gate Mechanism(2 posts)→
More from Research
- MIT's Neil Thompson: AI comes for tasks before jobs, and fixed costs favor big firms — bigdata · 2026-10-16
- UMD's PAIR framework bridges perception and action in VLA models via bridge tokens — Kaixi Feng · 2026-10-16
- CI-MSE offline metric hits -0.87 Spearman correlation with real robot task success vs weak raw MSE — chris_j_paxton · 2026-10-16
- Colossal Biosciences uses AI to decode genetic switches behind the extinct dodo — castrotech · 2026-10-16
- Colossal Biosciences uses AI to identify gene switches in dodo de-extinction push — castrotech · 2026-10-16
- Cheng Lou open-sources Handwired: hand-written weights turn code into training-free neural nets — dotey · 2026-10-16