Adaptive Color Grading
Trevor D. Canham, Abhijith Punnappurath, Michael S. Brown
eess.IV, cs.CV
2026-09-18
A 1,564-frame tonescale-threshold dataset plus KNN beats a static LUT, U-Net, and NILUT: ΔE00 2.40 and PSNR 36.77 on remainder→NYC, with chroma offsets held fixed.
Colorists grade by tonescale region: shadows, lights, blacks, and speculars get separate chromatic pushes so a flat frame reads as volume. Commercial tools either freeze those cuts or split by percentiles. A 3D LUT tuned on shot A collapses on shot B, and the blue that belonged in shade leaks into midtones.
End-to-end enhancers learn the whole retouch from input/output pairs. They rarely check whether any one operation, such as the region cut, is correct. Subjective style is a poor pixel-regression target.
The missing piece is a small set of knobs. If tonescale region thresholds (TRTs) track the image, the same regional tints can ride along.
The authors ship an open-source grader. mean(R,G,B) splits the tonescale into two overlapping pairs, dark/darkest and light/lightest, each with a threshold and a fixed-slope falloff. Users add CIELAB ab offsets per region, nested through a 17³ LUT.
They graded 782 shots from the HDR Videographic Survey, two frames each, 1,564 DCI 2K stills across 44 scenes in Toronto, Utah, Alaska, Seattle, London, and NYC. The creative brief is fixed: separate direct light from shade, and use the extreme bands to protect blacks, lamps, speculars, and sky. Reflectance mix-ups, haze, and long penumbras make the cuts ill-posed.
Models predict only the four TRTs. Chroma offsets are pinned: darkest {0,-10}, dark {0,10}, light {0,-10}, lightest {0,10}, roughly blue shade and yellow key. Four application-specific predictors:
U-Net, NILUT, and NamedCurves are trained as end-to-end controls on NYC/Night versus the remainder, both directions.
Tables report ΔE00 (lower better) and PSNR. KNN wins all four splits.
| Method | remainder→NYC ΔE00 / PSNR | NYC→remainder | remainder→Night | Night→remainder |
| U-Net | 2.96 / 34.90 | 3.83 / 33.15 | 2.99 / 36.22 | 2.81 / 36.19 |
| NamedCurves | 3.44 / 32.86 | 4.02 / 32.44 | 4.85 / 30.99 | 3.83 / 32.68 |
| NILUT | 2.65 / 35.46 | 2.86 / 35.80 | 1.62 / 40.84 | 2.56 / 36.87 |
| Fixed | 3.25 / 34.16 | 3.56 / 34.39 | 1.81 / 40.47 | 2.96 / 35.90 |
| MLP | 3.41 / 33.71 | 3.15 / 35.24 | 2.10 / 38.13 | 3.68 / 33.00 |
| KNN | 2.40 / 36.77 | 2.55 / 37.53 | 1.47 / 42.94 | 2.50 / 37.26 |
MLP and percentiles often lose to the static LUT. Correlation plots show why: fixed cuts are horizontal bands, percentiles swing with histogram mass, the MLP forms round clouds, KNN keeps the four bands separated. Qualitatively only KNN preserves the yellow/blue split; the rest wash toward a global cast. NILUT beats the static LUT on paper by staying close to the input.
This is a large empirical test of the "fixed stops or percentiles cut illumination" folklore. Percentiles are worst, fixed is tolerable, and only KNN consistently clears the static LUT. For camera pipelines and finishing tools the lesson is narrow and usable: label the knobs, then learn them. Training fits on a laptop CPU; KNN queries in 0.1 s, which is the right shape for on-device personalization.
The task is also narrower than real grading, because chroma is frozen at test time.
One annotator, one creative intent, and TRTs are only one parameter family in a colorist's desk. Chroma is not learned. End-to-end models do not see the tool's falloff functions, so the comparison is not fully symmetric. Frames were graded after a Resolve tone map into P3D65; phone JPEGs and LOG plates would need a new annotation pass. Local-region and motion-picture extensions are named, not run.