Validation-Centric AI-Assisted GPU Porting of a 250,000+ Line Legacy Weather Simulation Code
Tetsuya Hoshino, Masaya Kato, Kazuhisa Tsuboki, Daichi Mukunoki, Takahiro Katagiri, Toshihiro Hanawa
cs.DC
2026-08-13
A Nagoya University team used a CLI AI agent to port CReSS, a 250K-line Fortran weather simulator, to GH200 GPUs via a validation-centric workflow: all 162 target kernels passed element-wise numerical checks, a real typhoon simulation ran 5.1x faster, and five floating-point discrepancies were caught and fed back to developers.
HPC systems are moving to GPU-centric architectures, but the scientific programs behind climate and disaster research are often hundred-thousand-line Fortran code bases built on MPI/OpenMP, where a manual port is measured in years. The catch is that these are not ordinary legacy code bases. CReSS has been developed since 1998, and its reliability comes from two decades of comparison against observations. The goal is not to have an AI regenerate the model, but to move the existing implementation onto GPUs without losing the scientific validity accumulated over decades.
The difficulty is concrete. AI-generated changes must be checked against the behavior of the CPU original, and meaningful kernel inputs are produced by initialization, physics, and configuration-dependent execution paths, so random synthetic data cannot reproduce real simulation states. GPU execution also changes evaluation order and intrinsic-function implementations, which makes bitwise CPU/GPU disagreement the norm. Someone has to decide, for every difference, whether it is an acceptable numerical variation or a porting bug. Weather models are especially sensitive here: small differences accumulate over time and can eventually flip conditional branches.
The workflow has six phases: code meta-review, profiling, kernel extraction with CPU benchmark generation, kernel-level GPU transformation, integration, and performance validation. The AI agent handles repetitive artifact generation; humans define validation criteria, own the specifications, and interpret numerical differences. Three design choices carry the weight:
Passing at the kernel level is not enough. After integration, the full 360-step run must keep both pressure-perturbation metrics within 1e-4 of developer-provided reference values.
| Metric | Result | Context |
| Application speedup | 5.1x (1.88s vs 9.51s per step) | 72-thread Grace CPU baseline; HBM-to-CPU memory bandwidth ratio is about 8x |
| 162 kernels | all pass element-wise dump validation | the subset of 387 OpenMP regions executed by this typhoon scenario |
| Numerical discrepancies | 5 kernels, single-element | floating-point and intrinsic-function differences, not porting bugs |
| Memory-bandwidth utilization | 35-60% of GPU peak | profiled per kernel with Nsight |
| Development cost | 100 GPU node-hours over 3 months | manual ports of this scale are typically planned in months to years |
Three of the five discrepancies are threshold-sensitive branches: in bruntv.f90 the temperature computes to 233.16002 K on CPU versus 233.16000 K on GPU, straddling the 233.16 K threshold, so a latent-heat correction applies only in the GPU run. disptke.f90 differs by about 25 ulps, pointing to an intrinsic-function implementation difference in exp/log/sqrt amplified by cancellation. These findings went back to the CReSS developers; none of them could have been localized from application-level output alone.
For teams maintaining large scientific codes, three lessons transfer. First, the validation-centric workflow positions AI as an accelerator of a process humans could barely afford, automating per-kernel dump-based validation, not replacing the developer's judgment. Second, using compiler diagnostics as a variable-discovery oracle converts one compile flag into a large shift of failure cost. Third, externalizing workflow rules and recovery policies into persistent specifications keeps constraints alive across agent session boundaries. In the controlled comparison, prompt-only runs converged 3 out of 5 times while specification-based runs converged 5 out of 5; adding a rule to batch-check similar variables after one dump failure cut dump re-executions from as many as 7 down to 1-3.
The authors are candid. Snapshot-based kernel validation covers only the states captured at selected dump points and does not certify all control paths; during integration, a benchmark that omitted a branch executed only at intermediate timesteps passed its snapshot check and failed only after integration. A minimal dump for one test case, one timestep, and one MPI process exceeded 400 GB, so scaling beyond this scenario requires a dump-data lifecycle strategy. Unified Memory plus intentionally kernel-local revision trades performance for verifiability; explicit data management, asynchronous execution, and kernel fusion are all left as future work, which partly explains realizing 5.1x against an 8x bandwidth ratio. The 100 node-hours were human-supervised exploratory time, not unattended production cost. And the observed failure modes come from Claude Code Opus 4.5-4.6; the specific behaviors may change with future models, though the three cost categories they identify (runtime-state reconstruction, session-spanning context, cost-aware recovery) are not model-specific.