Spectral Subspace Rewiring Enhances Reasoning and Merging

Zhilong Zhang · hf · 2026-07-17

Key Points

This work proposes SAR (Subspace-Aligned Rewiring) to address two issues in the post-training of large models:

Suppressed reasoning performance: Standard full-parameter updates cause test-time scaling gains to saturate prematurely.
Interference between multi-domain training/model merging: Update directions conflict when mixing different capabilities.

Method

The authors discovered that updates effective for reasoning primarily reside within the spectral space of the base model. As a post-training editing method, SAR preserves this "spectral core" and removes interfering components in orthogonal directions, recovering and enhancing effective capabilities without retraining.

Results

Across various model families and scales, SAR extracts a compact reasoning core using only about 0.58% of total parameters.
Retains over 99% of post-training performance.
Enhances high-level exploration capabilities in mathematical reasoning.
Generalizes to agentic coding: improves performance on 6 out of 7 open benchmarks on custom models.
In mixed-domain training, SAR unleashes suppressed coding abilities while maintaining math reasoning and instruction following.
For model merging, it delivers better cross-domain generalization, surpassing previous merging baselines and even outperforming the best single-domain experts.

Original post →

Spectral Subspace Rewiring Enhances Reasoning and Merging

More from coding & agent