The new generation of heterogeneous CPU/GPU computer systems offer much greater computational performance but are not yet widely used for climate modeling. One reason for this is that traditional climate models were written before GPUs were available and would require an extensive overhaul to run on these new machines. In addition, even conventional “high–resolution’ simulations don’t provide enough parallel work to keep GPUs busy, so the benefits of such overhaul would be limited for the types of simulations climate scientists are accustomed to. The vision of the Simple Cloud-Resolving Energy Exascale Earth System (E3SM) Atmosphere Model (SCREAM) project is to create a global atmospheric model with the architecture to efficiently use GPUs and horizontal resolution sufficient to fully take advantage of GPU parallelism. After 5 yrs of model development, SCREAM is finally ready for use. In this paper, we describe the design of this new code, its performance on both CPU and heterogeneous machines, and its ability to simulate real-world climate via a set of four 40 day simulations covering all 4 seasons of the year.

We investigate a set of Energy Exascale Earth System Model Multi-scale Modeling Framework (E3SM-MMF) simulations that vary the dimensionality and momentum transport configurations of the embedded cloud-resolving models (CRMs), including unusually ambitious 3D configurations. Issues endemic to all MMF simulations include too much ITCZ rainfall and too little over the Amazon. Systematic MMF improvements include more on-equatorial rainfall across the Warm Pool. Interesting sensitivities to CRM domain are found in the regional time-mean precipitation pattern over the tropics. The 2D E3SM-MMF produces an unrealistically rainy region over the northwestern tropical Pacific; this is reduced in computationally ambitious 3D configurations that use 1024 embedded CRM grid columns per host cell. Trajectory analysis indicates that these regional improvements are associated with desirably fewer tropical cyclones and less extreme precipitation rates. To understand why and how the representation of precipitation improved in 3D, we propose a framework that dilution is stronger in 3D. This viewpoint is supported by multiple indirect lines of evidence, including a delayed moisture-precipitation pickup, smaller precipitation efficiency, and amplified convective mass flux profiles and more high clouds. We also demonstrate that the effects of varying embedded CRM dimensionality and momentum transport on precipitation can be identified during the first few simulated days, providing an opportunity for rapid model tuning without high computational cost. Meanwhile the results imply that other less computationally intensive ways to enhance dilution within MMF CRMs may also be strategic tuning targets.

We design a new strategy to load-balance high-intensity sub-grid atmospheric physics calculations restricted to a small fraction of a global climate simulation’s domain. We show why the current parallel load balancing infrastructure of CESM and E3SM cannot efficiently handle this scenario at large core counts. As an example, we study an unusual configuration of the E3SM Multiscale Modeling Framework (MMF) that embeds a binary mixture of two separate cloud-resolving model grid structures that is attractive for low cloud feedback studies. Less than a third of the planet uses high-resolution (MMF-HR; sub-km horizontal grid spacing) relative to standard low-resolution (MMF-LR) cloud superparameterization elsewhere. To enable MMF runs with Multi-Domain CRMs, our load balancing theory predicts the most efficient computational scale as a function of the high-intensity work’s relative overhead and its fractional coverage. The scheme successfully maximizes model throughput and minimizes model cost relative to precursor infrastructure, effectively by devoting the vast majority of the processor pool to operate on the few high-intensity (and rate-limiting) HR grid columns. Two examples prove the concept, showing that minor artifacts can be introduced near the HR/LR CRM grid transition boundary on idealized aquaplanets, but are minimal in operationally relevant real-geography settings. As intended, within the high (low) resolution area, our Multi-Domain CRM simulations exhibit cloud fraction and shortwave reflection convergent to standard baseline tests that use globally homogenous MMF-LR and MMF-HR. We suggest this approach can open up a range of creative multi-resolution climate experiments without requiring unduly large allocations of computational resources.