Contents Menu Expand Light mode Dark mode Auto light/dark mode
aws-cmaq documentation
Logo
aws-cmaq documentation

Contents:

  • 1. Create Single VM
    • 1.1. Create a VM from the AWS Web Console
    • 1.2. Create a VM using the AWS Command Line
    • 1.3. Run CMAQv5.4 on c6a.2xlarge
  • 2. Create a Parallel Cluster and run CMAQv5.4
    • 2.1. Build a Demo ParallelCluster
    • 2.2. Use ParallelCluster with Software and Data pre-installed on hpc7g.16xlarge
    • 2.3. Run CMAQ on hpc7g.16xlarge
    • 2.4. Run DESID CMAQ on hpc7g.16xlarge
    • 2.5. Modify the ParallelCluster to remove the lustre filesystem
    • 2.6. Create Cost Allocation Tags for Analysis using AWS Cost Explorer.
  • 3. Performance and Cost Optimization
    • 3.1. ParallelCluster Configuration
    • 3.2. CMAQv5.4 Benchmarks
    • 3.3. Slurm Compute Node Provisioning
    • 3.4. Benchmark Timings for CMAQv5.4 12US1 Benchmark
    • 3.5. Benchmark Scaling Plots for CMAQv5.4 12US1 Benchmark
    • 3.6. Cost Information
    • 3.7. Recommended Workflow for extending to annual run
  • 4. Developer Guide to install and run CMAQv5.4 on Single VM or Parallel Cluster
    • 4.1. Install CMAQv5.4+ on Single Virtual Machine Advanced (optional)
      • 4.1.1. Install Software and run CMAQv5.4 on c6a.xlarge for the 12km Listos Training Domain
      • 4.1.2. Install Software and run CMAQv5.4 on c7g-hpc7g for the 12km Listos Training Domain
      • 4.1.3. Install I/O API libraries that support HDF5
      • 4.1.4. Upgrade to run CMAQ on larger EC2 Instance
    • 4.2. Install CMAQv5.4 on ParallelCluster (optional)
      • 4.2.1. Configure Parallel Cluster
      • 4.2.2. Create the hpc7g.16xlarge pcluster
      • 4.2.3. Install CMAQ sofware and libraries on ParallelCluster version 3.6
      • 4.2.4. Install netCDF libraries that use HDF5 and support nc4 compressed files
      • 4.2.5. Install gh following these instructions
      • 4.2.6. Use gh authentication
      • 4.2.7. Run CMAQ using hpc7g.16xlarge compute nodes
      • 4.2.8. Run CMAQ using hpc7g.8xlarge compute nodes
      • 4.2.9. Install Input Data on ParallelCluster
  • 5. Post-process and QA
    • 5.1. Post-process CMAQ and Install R and Anacoda
      • 5.1.1. Scripts to run combine and post processing
      • 5.1.2. Install R, Rscripts and Packages
      • 5.1.3. Install Anaconda on the /shared/build directory
    • 5.2. QA CMAQ
      • 5.2.1. Quality Assurance
      • 5.2.2. Run m3diff to compare the output data for two runs that have different values for NPCOL
      • 5.2.3. Run an R script to create the box plots and spatial plots comparing the output of two runs
      • 5.2.4. Run Jupyter Notebook to analyze difference between with DESID Emissions and the base case (no emission reduction)
    • 5.3. Compare Timing of CMAQ Routines
      • 5.3.1. Parse timings from the log file
    • 5.4. Copy Output to S3 Bucket
      • 5.4.1. Copy Output Data and Run script logs to S3 Bucket
      • 5.4.2. Copy scripts and logs to /fsx
      • 5.4.3. Examine the output files
      • 5.4.4. Copy the output to an S3 Bucket
  • 6. Logout and Delete ParallelCluster
    • 6.1. Logout of cluster when you are done
    • 6.2. Delete Cluster
    • 6.3. Verify that the cluster was deleted
  • 7. Run AMET on a VM
    • 7.1. AMET on AWS
    • 7.2. Spin up a server using pre-installed AMET AMI
    • 7.3. Create Air Quality Plots using the AMET AQ Website
    • 7.4. Programs to create plots (74)
    • 7.5. Air Quality Observation Networks (48)
    • 7.6. Air Quality and Met Species (172)
    • 7.7. Method used to create plots
    • 7.8. Example plots using the aqExample database
    • 7.9. Load your own AQ model data to MariaDB
    • 7.10. Loading EPA’s EQUATES Database
    • 7.11. Example plots using the EQUATES 2002-2019 Projects in the amad_EQUATES database
    • 7.12. Create Met Plots using the AMET Met Website
    • 7.13. Met Observation Networks
    • 7.14. Load your own MET data to MariaDB
    • 7.15. Types of Errors Creating Plots and how to avoid them.
    • 7.16. How AMETv1.6 was installed (developers only)
  • 8. Additional Resources
    • 8.1. FAQ
    • 8.2. Free Training
    • 8.3. Another workshop to learn the AWS CLI 3.0
    • 8.4. Youtube video
    • 8.5. Intro to AWS for HPC People - HPC Tech Shorts
    • 8.6. Benchmarking
    • 8.7. Help Resources for CMAQ
    • 8.8. Computing on the Cloud References
    • 8.9. AWS Resources for the aws cli method to launch ec2 instances.
    • 8.10. Resources from AWS for diagnosing issues with running the Parallel Cluster
    • 8.11. Instructions on how to create Parallel Cluster Amazon Machine Image (AMI) from the command line
    • 8.12. ParallelCluster Update
    • 8.13. Use Elastic Fabric Adapter/Elastic Network Adapter for better performance
    • 8.14. VPC Management
    • 8.15. Using Cost Allocation Tags with ParallelCluster
  • 9. Future Work
    • 9.1. Future Work
  • 10. Contribute to this Tutorial
    • 10.1. Contribute to Pcluster-cmaq Documentation
Back to top

5.2. QA CMAQ#

Quality Assurance: Comparing the output of two CMAQ runs.

  • 5.2.1. Quality Assurance
  • 5.2.2. Run m3diff to compare the output data for two runs that have different values for NPCOL
  • 5.2.3. Run an R script to create the box plots and spatial plots comparing the output of two runs
  • 5.2.4. Run Jupyter Notebook to analyze difference between with DESID Emissions and the base case (no emission reduction)
Next
5.2.1. Quality Assurance
Previous
5.1.3. Install Anaconda on the /shared/build directory
Copyright © 2022, CMAS Center
Made with Sphinx and @pradyunsg's Furo
Last updated on 2026-06-18 16:23:10 +0000