SSNA: transferring knowledge from noise domains

量子位 · wechat · 2026-07-20

Semi-supervised learning from noise

This paper, Semi-Supervised Noise Adaptation (SSNA), asks a counterintuitive question: can a model transfer useful structure from a source domain made entirely of random noise?

Main idea

Instead of using real labeled source data, the method constructs a synthetic source domain by sampling Gaussian noise and assigning pseudo class indices. The model then learns:

a target encoder from a small number of labeled real target samples,
a noise projector that turns random vectors into separable class structure,
and a domain-alignment objective that matches the noise domain to the target domain in a shared representation space.

Key results

The paper reports that the proposed Noise Adaptation Framework (NAF) improves low-shot performance across multiple datasets:

On CIFAR-10/100, DTD-47, Caltech-101, with only 4 labels per class, NAF beats standard supervised training by large margins.
It also works on fine-grained benchmarks such as CUB-200, OxfordFlowers-102, StanfordCars-196.
On ImageNet-1K, with 100 labels per class, NAF reaches 37.10% Top-1, ahead of ERM.
On AGNews-4 with BERT, it also improves text classification.

What actually matters

The ablations show the benefit comes from separable structure in noise, not from the noise being “random” itself. If all noise classes collapse together, performance collapses too. More clearly separated synthetic classes generally help more, while simply increasing the number of noise samples is less important.

Why it matters

The work suggests that when real source data is unavailable due to privacy, copyright, or cost, structured synthetic noise can serve as a low-cost alternative source for transfer learning.

Original post →

SSNA: transferring knowledge from noise domains

More from Research