When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference
Ismail Erbas, Xavier Intes, Vikas Pandey
cs.AI, cs.LG, physics.optics, q-bio.QM
2026-09-04
On a frozen GRU, 4-bit state write-back lifts τ1/τ2 RMSE from 0.36/0.35 ns to 25.4/106.6 ns. Error feedback and direction memory restore accuracy without retraining.
Quantization is usually scored as “how many bits can we drop.” In a recurrent net the quantized state is stored and fed back, so the storage rule is part of the temporal computation. Mapping a trained network onto a coarser grid at deployment time sends it down a trajectory it never trained for. Fluorescence lifetime imaging (FLI) fits a noisy time-resolved decay to a short lifetime τ1 and a long lifetime τ2. A small warp in temporal shape blows up the integrated lifetime, which makes the task a sharp probe.
The model is Seq2SeqLite: a one-layer 32-unit GRU encoder–decoder, 6,627 parameters, 135 time bins in. Data are 1.6 million simulated decays, with 160,000 held out.
Recurrent-state write-back is the map from the state computed at step t to the value presented at t+1. Deterministic B-bit write-back rounds to the nearest representable level; a proposed update inside the half-step deadband leaves the stored value unchanged. The recurrent write margin compares that proposal with the quantizer half-step; a margin below 1 means the write will not land.
To separate storage from learning, checkpoints are frozen and only the write-back rule changes. Comparators include identity propagation, stochastic rounding, error feedback, k-bit residual memory, and direction memory, which counts same-sign sub-threshold proposals and steps one grid level when the counter fills. Independently trained 4-bit and 8-bit GRUs are swept across post-training precisions. The same freeze-and-swap protocol is repeated on an LSTM, with cell and hidden states targeted together or apart.
When QMem attaches state quantization at P2F, τ1/τ2 RMSE jumps from 1.40/3.03 ns at P2E to 25.37/106.59 ns, then P3 training recovers 0.48/0.55 ns. Freeze P2F and switch write-back to identity: 0.36/0.35 ns. Switch it back to deterministic 4-bit: 25.37/106.59 ns, roughly 70× and 300×. Sequence MAE stays 0.09, so pointwise reconstruction does not reveal the lifetime failure.
In the failing decoder, 99.59% of updates sit in the 4-bit deadband, 0.08 of 32 units change level per step, and 97.25% of steps change none. Median same-sign deadband runs last 132 of 134 steps. A native 4-bit model still changes 12.47% of decoder state per step, with median runs of 2 steps. Coarse grids are not the cause; long stretches of uncommitted updates are.
On that same frozen P2F net, error feedback recovers 0.36/0.37 ns, 2-bit residual memory 0.34/0.40 ns, and 3-bit direction memory 0.34/0.34 ns. An 8-bit reference GRU at 0.20/0.22 ns native becomes 1.89/2.60 ns under forced 4-bit, then 0.34/0.46 ns with error feedback. Widening a native 4-bit GRU to 8-bit raises occupied decoder levels from a median of 12 to 176.5 and RMSE from 0.35/0.40 ns to 0.43/0.57 ns.
On the LSTM, 4-bit on both states moves 0.239/0.254 ns to 3.857/1.327 ns. Cell-only 4-bit hits 8.158/1.292 ns; hidden-only hits 0.296/0.354 ns. The cell is more fragile, even though the hidden state has the larger deadband fraction.
Low-precision recurrence is a question of trajectory compatibility, not of how finely one stored value is represented. A learned solution is tied to the write-back interface it trained with; changing that interface at deploy time changes the dynamics. Direction memory shows that keeping “which way we kept trying to go” can be enough; the discarded magnitude does not always have to be stored. On-device RNNs, sensors, and imaging front-ends should spec the state-storage rule, not only the bit width.
The story is told on a 32-unit FLI encoder–decoder. It does not reach recurrent or SSM state inside a large language model. Matched training compares four memory allocations and does not rank operators across tasks. Identity propagation does not restore the LSTM's native 8-bit solution, so “finer” and “back on the trained interface” are different, and the paper does not hand out a recipe for arbitrary architectures. Lifetime is integrated from a reconstructed sequence and is unusually sensitive to temporal shape; other recurrent tasks may not amplify the same way.