SPS Transformer Splits State and Prediction
burny_tech · x · 2026-07-15
A shared post highlighted a new Transformer study. The author argues that current Transformers use the same hidden state to simultaneously handle two competing responsibilities: maintaining state and predicting the next token.
This work proposes splitting these roles into two independent streams:
- One dedicated to state representation
- One dedicated to token prediction
The person sharing the post noted that this direction is reminiscent of Parallel Speculative Sampling (PaSS) from years ago. They praised the work for continuing to explore the decoupling of "state representation learning" from "token prediction learning." The original post also claimed that this SPS Transformer performs better than standard Transformers.
Related event: SPS Transformer: Separating State and Prediction into Dual Streams(5 posts)→
More from Research
- Project APE finds verifier reliability drops when papers contain multiple errors — soumitrashukla9 · 2026-07-22
- Project APE says verifier costs fell about 90x in a year as Chinese open models lead — soumitrashukla9 · 2026-07-22
- OpenAI-linked paper says capability RL can make models more reward-seeking — MariusHobbhahn · 2026-07-22
- Project APE builds its verifier benchmark from 100 AI-written papers with injected errors — soumitrashukla9 · 2026-07-22
- Paper proposes a CRED taxonomy and benchmark to measure research-error detectors — soumitrashukla9 · 2026-07-22
- OpenAI says long-horizon models need safety and alignment checks across full action sequences — rhiever · 2026-07-22