DySENSE: Dynamic Spatiotemporal ENergy Synthesis and Emulation with Satellite Imagery
Kailai Sun, Zijie Gao, Mingyi He, Can Rong, Alok Prakash, Baoshen Guo, Shenhao Wang, Jinhua Zhao
KDD 2027
cs.CV, cs.AI
2026-11-13
DySENSE pairs satellite tiles with monthly energy maps. With about 25% real labels, synthetic data raises mIoU by up to 8.7 points and cuts CVRMSE by 9-18 points on four cities.
Urban building energy modelling (UBEM) usually answers how much energy an existing city uses. Planners need a counterfactual: change the land-use mix, or keep the block fixed and switch the month and climate, and read the new monthly energy map.
Statistical models and physics simulators stay tied to cities that already exist. Monthly energy lined up with satellite imagery is scarce, because electricity and gas disclosures run into privacy and cost. Generative models can emit satellite images, one-dimensional meter series, or a static annual energy layer. A two-dimensional map that moves with month and climate, on a fixed urban form, is the missing piece.
DySENSE separates slow morphology from fast energy and fits P(x, e | cs, cd). x is a 2 km satellite tile and e is the aligned energy map. The static prompt cs records location and land-use shares. A frozen CLIP encoder turns it into 77 tokens. The dynamic prompt cd records the month and five climate variables: maximum and minimum temperature, precipitation, vapor pressure, and surface shortwave radiation. A frozen RemoteCLIP encoder, pretrained on remote-sensing captions, embeds that prompt. In the ablation, RemoteCLIP beats CLIP on both generation and downstream prediction.
Energy maps are sparse and heavy-tailed, so they do not go through the photo autoencoder. The image branch keeps a pretrained VAE. A separate Energy VAE is trained for the energy map. Both latents are 64×64×4. The denoiser is a U-ViT, a diffusion network built from Transformer blocks. Image tokens and energy tokens form one sequence and share a single noise draw and a single timestep, so denoising has to learn their spatial link. Text conditions stay at timestep 0.
Monthly rollout does not need a second network. Fix the image latent, denoise only the energy latent, and replace cd. The morphology stays put while month and climate change, and the same tile yields 12 energy maps.
Targets are not continuous kBtu. Electricity and gas are summed on a 100 m grid, compressed with log1p, and cut into 10 quantile bins. Class 0 is background or no positive energy. The other nine bins are equal-frequency levels.
MUSE2 covers New York City, Washington, D.C., Norfolk, and Busan. Mapbox tiles are 512×512. Three specialists drop tiles with large empty blocks, leaving 2,565 spatial tiles: 579, 490, 500, and 996. Monthly pairs are balanced to 23,880, with 5,880 in Norfolk and 6,000 in each other city. U.S. climate is Daymet at about 1 km. Busan temperatures are TerraClimate at about 4.6 km; precipitation and radiation come from the Korea Meteorological Administration. The climate grids do not match.
Generated images score FID 36.74, LPIPS 0.56, and PSNR 11.52. Pixels do not stick to the real tile. Alignment is tested with a downstream predictor: a pair that only looks plausible, and sits in the wrong place, should not help.
The test set is a fixed 20% of real tiles. The generator and the predictors share one real training split. Controls are real data alone, an equal count of duplicated samples, and an equal count of Gaussian-noised samples. Static Mix adds synthetic static image-energy pairs. Dynamic Mix also adds month-varying samples. mIoU is the mean intersection-over-union over energy bins. CVRMSE is the aggregate calibration error from ASHRAE Guideline 14. Lower is better.
| Predictor | Real only Acc / mIoU / CVRMSE | Dynamic Mix |
| Yap et al. | 0.5343 / 0.3687 / 33.58% | 0.6217 / 0.4555 / 15.31% |
| Mayer et al. | 0.5887 / 0.4212 / 24.27% | 0.6274 / 0.461 / 15.26% |
| Sun et al. | 0.5372 / 0.3708 / 30.57% | 0.5999 / 0.4327 / 16.62% |
The paper ties the Yap row to a 25% real-training split. Accuracy rises 8.7 points, mIoU moves from 0.3687 to 0.4555, CVRMSE falls 18.3 points, and NMBE, the normalized mean bias, falls from 6.02% to 2.82%. Mayer gains 3.9 accuracy points and 9.0 CVRMSE points. Sun moves from 0.5372 to 0.5999 accuracy and from 30.57% to 16.62% CVRMSE. The abstract's 8.7% accuracy is the Yap lift. The 9% to 18% CVRMSE band is the spread across the three predictors. Duplication barely moves the numbers. Gaussian noise lifts Yap accuracy only from 0.5343 to 0.5462.
Static Mix matters more when labels are thin. At a 6.25% training ratio, mIoU rises by about 7 points and CVRMSE falls from 82.3% to 58.4%. At 12.5%, mIoU rises by about 6.4 points. With the full real set, that margin shrinks.
Monthly distributions sit close to the ground truth: KL 0.0024, JS 0.0006, total variation 0.0181. After dropping dominant class 0, the figures are 0.0085, 0.0021, and 0.0496. Mean Pearson correlation of the monthly curves is 0.96, with Busan at 0.93 and New York City at 0.98.
The predictors remain three published models. DySENSE supplies paired synthetic samples, and all three improve, so the gain is not an artifact of one backbone. A planner can pin the morphology, edit month and climate, and see which month carries the peak. MUSE2 and the code are public.
The generator takes about 220 GPU-hours, the downstream models more than 600, and sampling about 60, on 10 L40S GPUs with 48 GB each. That budget fits a benchmark and a scenario generator. It does not fit a lightweight query tool.
Coverage is four metropolitan areas. The paper lists what is missing: more climate zones, data-scarce cities in developing regions, and an external check against utility audits or smart meters. Operational carbon would also need grid carbon intensity, which is not modelled here.
Cities that do not exist yet appear only in the sampler. Every scored test tile comes from these four real cities. No physics engine and no new-city measurement checks an unseen land-use mix. PSNR 11.52 also means the imagery is a poor planning basemap.
Energy is 10 quantile bins, so mIoU is a segmentation score. CVRMSE needs those bins mapped back to continuous energy. The mapping sits in the appendix, and the main text does not report its error. Class 0 pulls KL and JS down. Foreground total variation near 0.05 is the distance that describes the energy bins themselves. On Yap, CVRMSE goes from 33.58% to 16.91% with Static Mix, then to 15.31% with Dynamic Mix. The monthly prompt adds a further cut. Most of the abstract's error drop is already in the static pairs.