Quantization Breaks RNN Memory: Error Feedback Restores GRU/LSTM Accuracy Without Retraining
Ismail Erbas · hf · 2026-09-07
This work identifies a concrete failure mode in quantized recurrent inference: state write-back rules suppress small updates, degrading temporal memory of GRUs and LSTMs at low precision.
The authors propose error feedback and residual memory mechanisms that compensate for the suppressed updates, restoring accuracy without any retraining across GRU and LSTM architectures—a practical reference for on-device recurrent deployment.
More from Infra
- AWS on agent prompt caching: explicit + implicit patterns to slash token costs — AWS ML Blog · 2026-10-27
- Thieves spoofing GPS steal $100M+ in data center cargo since June — Ars Technica AI · 2026-10-27
- Lam Research: weekly token usage up ~250x in 18 months — Beth_Kindig · 2026-10-27
- AWS shows how to build enterprise agentic retrieval on Bedrock with CDK — AWS ML Blog · 2026-10-27
- Encrypted GPT-2 speedrun challenges FHE inference: best is 3.031 s/token, goal sub-1s — tensorqt · 2026-10-27
- Open-source Latch compiles PyTorch into training megakernels, 2.75x faster on H100 — sguada · 2026-10-27