Radar object classification: point accumulation beats sequence modeling by 40x

bruno_pinto90 · reddit · 2026-10-01

The author extended a single-scan radar object classifier on RadarScenes to accumulate observations across a tracked object's history, finding that most of the temporal gain comes from having more points, not from fancier sequence models.

Background & method

A RadarScenes instance averages only 2.9 radar points — extremely sparse; single scans can't capture temporal cues like RCS fluctuation or pedestrian micro-Doppler from limb motion.
Baseline: a DeepReflecs encoder (RadarConf 2021, PointNet-style) on single scans, 5 classes, macro F1 = 0.7370.
Using trackid, he builds a causal, N=20, stride-1 sliding-window buffer per track: odometry-corrected coordinates recentered on the object centroid, all 4 sensors share one buffer, each scan encoded once by a frozen encoder and cached; a fusion head concatenates a causal GRU's hidden state with an order-invariant pooled embedding.

Results (N=20, all 4 sensors)

20-scan point pooling: 0.8613 (+0.1243)
Causal GRU: 0.8895 (+0.0282 more)
GRU + pooled fusion: 0.8897 — no measurable gain
Ablations: larger GRUs, a Transformer, a state-space model, point-level self-attention all land in 0.86–0.89; end-to-end fine-tuning of the frozen encoder slightly hurts (-0.002).

Takeaway: pooling with zero notion of scan order recovers 4–40x every other improvement; once the per-scan embedding is fixed, no sequence model adds more than 0.003. Sparsity remains the dominant limitation — improving per-scan/point-level representations beats searching over sequence architectures. Full report with all ablations is open-sourced on GitHub.

Original post →

Radar object classification: point accumulation beats sequence modeling by 40x

More from Embodied