Resolution-Agnostic Neural Operators for Multi-Rate Sparse-View CT
Aujasvit Datta, Jiayun Wang, Asad Aali, Anima Anandkumar
eess.IV, cs.CV
2025-12-13
CTO, a dual-domain DISCO neural operator from Caltech, reconstructs sparse-view CT at 9–72 angles with one model: +3.4 dB over a matched CNN on AAPM, 500× faster than diffusion.
Sparse-view CT reconstructs a slice from far fewer X-ray projections, which cuts dose and scan time. The inverse of the Radon transform becomes ill-posed, and filtered back projection (FBP) leaves streak artifacts.
Learned reconstructors cleaned up those artifacts and then locked themselves to one sampling rate. A CNN kernel is sized for a fixed grid; drop from 72 views to 18 and the receptive field no longer covers the same physical neighborhood. LEARN trained only at 72 views scores 32.32 dB there and 4.16 dB at 18 views. Clinics change protocols by organ. Training a separate network per setting does not scale.
CTO treats reconstruction as a map between function spaces. The sinogram p(θ, r) and the image x(x, y) are functions; a pixel grid is just a sampling. Change the sampling rate and the same operator still applies.
The skeleton is an unrolled network with 3 cascades. Each cascade does a physics (data-consistency) step plus an image-domain neural operator NOi. A sinogram-domain operator NOs sits in front. The CNN prior is replaced by DISCO, discrete-continuous convolution.
DISCO parameterizes the kernel in continuous space with a piecewise-linear basis and discretizes it to the current grid at runtime. Raise the resolution and the kernel still covers the same physical neighborhood, which a CNN kernel does not. That is discretization convergence: as the grid refines, approximation error stays bounded and goes to zero.
Two CT-specific choices.
NOs runs in polar coordinates (r, θ), with a spatial branch and a frequency branch. The frequency branch takes a 1D Fourier transform along the detector axis r, then a U-shaped DISCO block (UDNO). That matches the Fourier slice theorem: the 1D spectrum of a projection is a polar slice of the object's 2D spectrum. The two branches are averaged. Along θ the model uses flipped circular padding, because a projection at θ+180° is the r-flipped projection at θ. Rotating the object then becomes a shift along θ, so the operator is rotation-equivariant.
NOi is the same UDNO on a Cartesian grid, restoring local high-frequency detail. The data-consistency term still uses the adjoint of the masked Radon operator, so the physics is kept.
Every learned method is trained on the same multi-rate mix: 9, 18, 36, and 72 views sampled uniformly per batch. On AAPM abdomen CT (526 test slices), CTO leads the matched unrolled CNN and the diffusion baselines:
| Method | 18-view PSNR | 36-view | 72-view | 18-view RMSE (HU) |
| FBP | 13.14 | 13.36 | 13.82 | 632.58 |
| DPS (diffusion) | 25.68 | 29.13 | 31.64 | 149.97 |
| GloReDi | 29.47 | 31.74 | 32.96 | 96.89 |
| Unrolled CNN | 29.14 | 31.76 | 33.35 | 100.65 |
| CTO | 31.57 | 35.06 | 37.88 | 75.97 |
Versus the matched CNN the gaps are 2.43 / 3.30 / 4.53 dB at 18 / 36 / 72 views, about 3.4 dB on average. Versus DPS the abdomen gap is about 6 dB. Inference is 0.065 s on an A100; DPS takes 51.58 s and ALD 32.72 s, so at least 500× faster, with no per-rate hyperparameter search.
On 10,000 kidney test slices, CTO reaches 36.17 / 38.76 / 40.97 dB at 18 / 36 / 72 views against 32.56 / 34.37 / 35.60 for the CNN, about 4 dB. Zero-shot super-resolution holds: 256→512 in image space is +3 dB over the CNN; 72×672→144×1344 in sinogram space is +3.7 dB.
Dropping NOs costs 4.82 dB, dropping NOi 3.86 dB. The frequency branch is worth 2.7 dB. Circular padding adds 0.83 dB at 24 views (33.00 vs 32.17). Train on kidney, test on abdomen: +9.8 dB over the CNN. Poisson noise at I0=10^6: +3.06 dB. On LIDC-IDRI lung CT the gaps versus the CNN are 1.54 / 2.47 / 3.29 dB at 18 / 36 / 72 views.
For medical reconstruction, this replaces "one protocol, one checkpoint" with one operator over a family of protocols. Scan settings change; four weight files do not. At inference the DISCO kernel is pre-discretized, so FLOPs match a convolution. The method is two orders of magnitude faster than diffusion. That is the regime a scanner can actually run.
For neural-operator work, global Fourier layers are a poor fit here: every pixel mixes with the whole image and edges wash out. DISCO keeps compact support, CNN-level cost, and discretization convergence. Code is at neuraloperator/sparsect.
The skeleton is incremental. Unrolling, dual-domain processing, and data consistency already exist. What changed is the prior: discrete convolution becomes a function-space operator, plus rotation-equivariant padding on the sinogram. Multi-rate co-training alone already repairs the worst overfitting (LEARN at 18 views goes from 4.16 dB single-rate to 25.51 dB joint). CTO then widens the gap on top of that co-training.
Future work names 3D CT, more clinical data, uncertainty, and broader scanner protocols. Experiments assume parallel-beam Radon geometry; real scanners are mostly fan-beam or cone-beam. Sparse views are uniformly subsampled from fully sampled scans, not acquired sparse on a machine.
The super-resolution "ground truth" is bilinearly upsampled, so those numbers are optimistic. The unrolled CNN does not have exactly the same parameter count, so the 3.4 dB is not a pure function-space ablation. Diffusion on kidney was scored on 500 slices for cost reasons, not the 10,000-slice table. Under per-rate training the lead shrinks: at 18 views CTO is 33.94 dB versus 32.35 for the CNN, only +1.59 dB. At 9 views, the hardest sparsity, CTO is only 0.75 dB above the strongest single-pass method GloReDi (27.87 vs 27.12). There is no radiologist reader study. PSNR, SSIM, and HU RMSE are not diagnostic utility.