Training-Adaptive Convolutional Sparse Coding via Information Bottleneck for Robust Visual Representation
Meng'en Qin, Yinchen Liu, Mingxuan Cui, Youlu Xing
cs.CV, cs.LG, stat.ML
2026-09-17
TA-CSC trains CSC’s λ inside unfolded FISTA. Clean ImageNet Top-1 hits 72.53%; adapting λ on 100 unlabeled samples lifts CIFAR-10-C Gaussian accuracy from 44.43% to 68.23%.
Most vision nets optimize the final task loss and never set an explicit knob for how hard intermediate features should be compressed. The information bottleneck (IB) view wants a representation T that drops input bits irrelevant to the label Y while keeping the bits that matter. Miss that balance and the net either leaks task information layer by layer or carries nuisance signal that collapses under input corruption.
Convolutional sparse coding (CSC) already has such a knob: a reconstruction term keeps the signal, an ℓ1 term shrinks complexity, and the sparsity coefficient λ sets how hard to squeeze. Prior CSC-in-the-net work (ML-CSC, SCN, SDNet) still treats λ as a hand-picked constant, one value per layer. That cannot learn the IB trade-off, and it cannot retune compression when the input is corrupted.
TA-CSC unfolds CSC with FISTA, two iterations by default. Each step takes a reconstruction gradient, applies a soft-threshold at λ/L, then a Nesterov extrapolation. Because λ sits in the threshold, a hypergradient can be backpropagated through the unrolled steps and updated with the convolutional dictionary and the rest of the net. Softplus keeps λ non-negative.
The training loss is the task loss plus the mean ℓ1 of each layer’s sparse code, with γ=0.001. The task term pushes sufficiency; the sparsity term pushes compactness. Their pull is the paper’s IB trade-off.
After source training, the backbone is frozen and λ is adapted on a small unlabeled set of corrupted samples. The adaptation loss is relative reconstruction error divided by λ: larger λ suppresses redundancy, while a blown-up relative error penalizes over-compression. Batch-norm statistics can be refreshed on corrupted batches.
TA-CSC-18 replaces only the first convolution of ResNet-18; TA-CSC-18all replaces every conv layer. CIFAR runs on one 2080Ti, ImageNet on four 3090s.
Clean Top-1:
| Method | CIFAR-10 | CIFAR-100 | ImageNet-1K | ImageNet speed |
| ResNet-18 | 95.54% | 77.82% | 68.98% | 2100 n/s |
| SDNet-18 | 95.20% | 78.31% | 69.47% | 1800 n/s |
| TA-CSC-18 | 96.18% | 79.63% | 71.12% | 1689 n/s |
| TA-CSC-18all | 97.65% | 80.76% | 72.53% | 153 n/s |
Corruptions are averaged over five severities. On CIFAR-10-C Gaussian noise, ResNet-18 scores 44.43%; TA-CSC-18all without adaptation already reaches 53.98%, and unlabeled λ adaptation lifts it to 68.23%. The ImageNet-C Gaussian numbers are 22.73% → 27.22% → 30.93%. The same adaptation also beats per-sample λ tuning on SDNet-18 (64.92% / 29.16% on the two Gaussian splits). Shot, speckle, and impulse noise move in the same direction. Adapted λ rises monotonically with corruption severity.
Ablations: 2→8 FISTA steps take CIFAR-10 from 96.18% to 96.93%; 50→500 adaptation samples add a little more robustness. Defaults stay at 2 steps and 100 samples because the extra compute is not worth it. During training λ starts small and grows, stays smaller in early layers and larger near the head, and spikes at ResNet’s four width expansions.
If the goal is an explicit “how much to compress” knob inside a classifier, this welds that knob into a differentiable CSC layer. Corruption can be met without labels and without retraining the backbone: only λ moves. Clean accuracy does not pay for the robustness; CIFAR-10 even rises from 95.54% to 97.65%.
The bill is equally clear. TA-CSC-18all on ImageNet uses 88.6 GB and 153 n/s against ResNet-18’s 24.1 GB and 2100 n/s. Replacing every conv layer is expensive. TA-CSC-18 is the more usable compromise: 71.12% clean ImageNet, 1689 n/s, and 66.63% on CIFAR Gaussian after adaptation.
The authors already flag the main gaps: ResNet and classification only, and the link from learned λ to information compression is empirical, not a tight IB proof. No Transformer, detection, or segmentation comparison. The CIFAR-10 jump from 95.54% to 97.65% is large enough to ask whether the training recipe favors CSC variants; no multi-seed table is given. The unlabeled adaptation still wants 100 target-domain corrupted images, which is not a free lunch in a true test-time-unknown setting.