LSA: learnable spectral activations beat SIREN and FINER on INR reconstruction
orlitany · x · 2026-10-23
Tamir Shor, Or Litany and Alex Bronstein released their NeurIPS paper "Learnable Spectral Activations" (LSA) with code.
- Idea: Prior INR work improves fitting by changing which frequencies the network can access (coordinate encodings, periodic nonlinearities). LSA argues signals with localized structure also need efficient composition of frequencies into multi-harmonic responses, so it replaces fixed neuron-level nonlinearities with a residual truncated Fourier series whose harmonic amplitudes are learned.
- Mechanism: No larger function class; instead it factorizes representation differently—linear weights select features while activation coefficients shape the spectrum, updated by separate gradients, making spectral tuning a more direct subproblem.
- Theory: This concentrates signal energy in leading NTK eigenmodes, improving optimization.
- Results: Gains across audio, image, NeRF and neural acoustic fields; gradient-only image reconstruction hits PSNR 19.95 dB vs 15.25 for SIREN and 13.48 for FINER.
More from Research
- Sophont's Holon-S1 predicts genes and biomarkers from routine pathology slides — iScienceLuvr · 2026-10-23
- Glasgow study: raised eyebrows and slow blinks make speakers look confident — Limor_Raviv · 2026-10-23
- Microsoft's step-by-step AI diagnosis research lands Science cover — erichorvitz · 2026-10-23
- IFM open-sources RL360, the agentic RL infra behind K2 Horizon's training — rupspace · 2026-10-23
- Wayfarer discovers RL options from a single experience stream, SOTA on hard Atari games — MarlosCMachado · 2026-10-23
- PhysVidBench accepted at EMNLP 2026: do video gen models grasp physical commonsense? — aykuterdemml · 2026-10-23