AutoCalibration

OpenSource tools from the E3SM Autotuning team

This is a working repository for the "Autotuning" Epic, started in FY21 under the NGD Software and Algorithms portfolio. The project is working to add some automation to the GCM tuning process using UQ and ML algorithms.

POCs are Benjamin M. Wagman, Lyndsay Shand, and Andy Salinger (formerly Drew Yarger and K. Chowdhary).

• The file "requirements.txt" sets up the Python packages and environment that we use for autotuning, resulting in the package versions described by "autotuning_environment.yaml". Our code depends on the "tesuract" and "clif" packages built by K. Chowdhary as specified in "requirements.txt".

• The "dakota_e3sm/" folder contains the code and configuration for running the perturbed parameter ensembles (PPEs) in our work.

• "src_v2/data/": preprocessed data used in Yarger et al. (2024) -- time-averaged spatial fields from the PPE, observations, and miscellaneous model runs

• The "src_v2/" folder contains code for the surrogate construction, optimization, and comparison of autotuning and default parameter sets that correspond to Yarger et al (2024) using E3SMv2.

  • "src_v2/models/": in "surrogate_latest.py", code that creates surrogate and optimizes parameters, depending on code in "preprocessing.py", "postprocessing.py", and "optimization.py". The file "config_latest.yaml" contains parameters for the surrogate (for example, which fields to use) so that one can construct a surrogate and optimize parameters using "python surrogate_latest.py config_latest.yaml".
  • "src_v2/mcmc/": code that runs Markov Chain Monte Carlo based on the optimization function set up in "config_mcmc.yaml", so that one can construct a surrogate, optimize parameters, and run MCMC using "python mcmc.py config_mcmc.yaml".
  • "src_v2/eda/": code for exploratory data analysis of the surrogate, perturbed parameter ensemble, and optimization results. This folder contains scripts to process the results after creating a surrogate, and is mostly based in R instead of Python.
    • "src/eda/paper_plots": code that recreates figures in Yarger et al. (2024)

• The "src_v3/" folder contains code for the surrogate construction, optimization, and figure generation that correspond to Wagman et al (2026) using E3SMv3.

  • "src_v3/models/": in "surrogate_latest.py", code that creates surrogate and optimizes parameters, depending on code in "preprocessing.py", "postprocessing.py", and "optimization.py". The file "config_latest.yaml" contains parameters for the surrogate (for example, which fields to use) so that one can construct a surrogate and optimize parameters using "python surrogate_latest.py config_latest.yaml".
    • Current config_surrogate.yaml and config_optimization.yaml configurations correspond to Wagman et al. (2026) and are set up to run with the following data on zenodo:
  • "src_v3/models/": in "surrogate_latest.py", code that creates surrogate and optimizes parameters, depending on code in "preprocessing.py", "postprocessing.py", and "optimization.py". The file "config_latest.yaml" contains parameters for the surrogate (for example, which fields to use) so that one can construct a surrogate and optimize parameters using "python surrogate_latest.py config_latest.yaml".
  • "src_v3/mcmc/": code that runs Markov Chain Monte Carlo based on the optimization function set up in "config_mcmc.yaml", so that one can construct a surrogate, optimize parameters, and run MCMC using "python mcmc.py config_mcmc.yaml".
  • "src_v3/history matching/": code that runs history matching -like sampling procedure or additional sampling described in Section 2.2 of Wagman et al (2026).
  • "src_v3/eda/": code for exploratory data analysis (EDA) of the surrogate, perturbed parameter ensemble, and optimization results. This folder contains scripts to process the results after creating a surrogate and miscellaneous data used in EDA.
  • "src_v3/sigma_temporal": code to compuute temporal variability used in optimization step.

• "plots_v3/": folder that contains R and python scripts to recreate the figures in the Wagman et al. (2026) preprint. Preprocessed data located on zenodo ([https://zenodo.org/records/21284943](Wagman 2025)).

Follow these steps to fit the surrogate and run the optimization code:

Step 1. Create and load the auto_paper conda env (see auto_paper_env.sh)

Step 2. Download the data from zenodo ([https://zenodo.org/records/22015064](Wagman et al., 2026))

Step 3. Use a compute node. Run the command:

'export OPENBLAS_NUM_THREADS=1'

Step 4. Change the data path in src/models/surrogate_and_optimization/config_surrogate.yaml and csrc/models/surrogate_and_optimization/onfig_optimization.yaml

default: './data/'

Step 5. Fit surrogate (run in src/models/surrogate_and_optimization/): 'python surrogate.py config_surrogate.yaml'

default saves to: './surrogate_models/'

Step 6. Update surrogate path in src/models/surrogate_and_optimization/config_optimization.yaml

default: '../../../surrogate_models'

Step 7. Run optimization (in src/models/surrogate_and_optimization/):

'python optimization.py config_optimization.yaml'

References:

Yarger, D., Wagman, B. M., Chowdhary, K., & Shand, L. (2024). Autocalibration of the E3SM version 2 atmosphere model using a PCA-based surrogate for spatial fields. Journal of Advances in Modeling Earth Systems, 16, e2023MS003961. https://doi.org/10.1029/2023MS003961

Wagman, B. M., Collins, G., Shand, L. and Harrop, B. E. (2026). Calibration of E3SMv3 for Low or High Equilibrium Climate Sensitivity using a Machine Learning Surrogate Model. Under Review.

Wagman, B. (2025, October). Merged e3smv3 ppe, surrogate predictions, and coarsened reference data. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.17429140 doi: 10.5281/zenodo.17429140

Wagman, B., Shand, L., & Collins, G. (2026, September). Surrogate and optimization data for e3smv3. Zenodo. Retrieved from https://doi.org/10.5281/zenodo.22015064 doi: 10.5281/zenodo.22015064