NTU paper: decision info lives in middle layers, last six layers removable loss-free

Minh-Anh Nguyen · hf · 2026-11-13

A paper by Minh-Anh Nguyen (Nanyang Technological University) on converting LLMs into efficient System One decision models.

Key findings

The usual transfer approach swaps the language head for a decision head but keeps the full backbone, so every decision pays for all layers
Decision information concentrates in intermediate layers: early layers build essential features, and the final six layers can be removed without loss
Independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities

Method (NESTONE)

Jointly trains a full model with prefix sub-models that share its final layers and decision head via self-distillation
A lightweight gate reads the hidden state at each exit to decide whether more computation is needed, letting one model run at multiple depths

Results: across seven decision benchmarks it matches or beats independently trained models at every depth, with better accuracy-efficiency trade-offs via adaptive computation. Efficiency comes not from smaller backbones but from shared computation across depths.

Original post →

NTU paper: decision info lives in middle layers, last six layers removable loss-free

More from Models