Contents Menu Expand Light mode Dark mode Auto light/dark mode
aws-cmaq documentation
Logo
aws-cmaq documentation

Contents:

  • 1. Create Single VM
    • 1.1. Create a VM from the AWS Web Console
    • 1.2. Create a VM using the AWS Command Line
    • 1.3. Run CMAQv5.4 on c6a.2xlarge
  • 2. Create a Parallel Cluster and run CMAQv5.4
    • 2.1. Build a Demo ParallelCluster
    • 2.2. Use ParallelCluster with Software and Data pre-installed on hpc7g.16xlarge
    • 2.3. Run CMAQ on hpc7g.16xlarge
    • 2.4. Run DESID CMAQ on hpc7g.16xlarge
    • 2.5. Use ParallelCluster with Software and Data pre-installed for c6a.48xlarge
    • 2.6. Run CMAQ on c6a.48xlarge
  • 3. Performance and Cost Optimization
    • 3.1. ParallelCluster Configuration
    • 3.2. CMAQv5.4 Benchmarks
    • 3.3. Slurm Compute Node Provisioning
    • 3.4. Benchmark Timings for CMAQv5.4 12US1 Benchmark
    • 3.5. Benchmark Scaling Plots for CMAQv5.4 12US1 Benchmark
    • 3.6. Cost Information
    • 3.7. Recommended Workflow for extending to annual run
  • 4. Developer Guide to install and run CMAQv5.4 on Single VM or Parallel Cluster
    • 4.1. Install CMAQv5.4+ on Single Virtual Machine Advanced (optional)
      • 4.1.1. Install Software and run CMAQv5.4 on c6a.xlarge for the 12km Listos Training Domain
      • 4.1.2. Install Software and run CMAQv5.4 on c7g-hpc7g for the 12km Listos Training Domain
      • 4.1.3. Install I/O API libraries that support HDF5
      • 4.1.4. Upgrade to run CMAQ on larger EC2 Instance
    • 4.2. Install CMAQv5.4 on ParallelCluster (optional)
      • 4.2.1. Configure Parallel Cluster
      • 4.2.2. Create the hpc7g.16xlarge pcluster
      • 4.2.3. Install CMAQ sofware and libraries on ParallelCluster version 3.6
      • 4.2.4. Install netCDF libraries that use HDF5 and support nc4 compressed files
      • 4.2.5. Install gh following these instructions
      • 4.2.6. Use gh authentication
      • 4.2.7. Run CMAQ using hpc7g.16xlarge compute nodes
      • 4.2.8. Run CMAQ using hpc7g.8xlarge compute nodes
      • 4.2.9. Install Input Data on ParallelCluster
  • 5. Post-process and QA
    • 5.1. Post-process CMAQ and Install R
      • 5.1.1. Scripts to run combine and post processing
      • 5.1.2. Install R, Rscripts and Packages
    • 5.2. QA CMAQ
      • 5.2.1. Quality Assurance
      • 5.2.2. Run m3diff to compare the output data for two runs that have different values for NPCOL
      • 5.2.3. Run an R script to create the box plots and spatial plots comparing the output of two runs
    • 5.3. Compare Timing of CMAQ Routines
      • 5.3.1. Parse timings from the log file
    • 5.4. Copy Output to S3 Bucket
      • 5.4.1. Copy Output Data and Run script logs to S3 Bucket
      • 5.4.2. Copy scripts and logs to /fsx
      • 5.4.3. Examine the output files
      • 5.4.4. Copy the output to an S3 Bucket
  • 6. Logout and Delete ParallelCluster
    • 6.1. Logout of cluster when you are done
    • 6.2. Delete Cluster
    • 6.3. Verify that the cluster was deleted
  • 7. Additional Resources
    • 7.1. FAQ
    • 7.2. Free Training
    • 7.3. Another workshop to learn the AWS CLI 3.0
    • 7.4. Youtube video
    • 7.5. Intro to AWS for HPC People - HPC Tech Shorts
    • 7.6. Benchmarking
    • 7.7. Help Resources for CMAQ
    • 7.8. Computing on the Cloud References
    • 7.9. AWS Resources for the aws cli method to launch ec2 instances.
    • 7.10. Resources from AWS for diagnosing issues with running the Parallel Cluster
    • 7.11. Instructions on how to create Parallel Cluster Amazon Machine Image (AMI) from the command line
    • 7.12. ParallelCluster Update
    • 7.13. Use Elastic Fabric Adapter/Elastic Network Adapter for better performance
    • 7.14. VPC Management
    • 7.15. Using Cost Allocation Tags with ParallelCluster
  • 8. Future Work
    • 8.1. Future Work
  • 9. Contribute to this Tutorial
    • 9.1. Contribute to Pcluster-cmaq Documentation
Back to top
Edit this page

1. Create Single VM#

Elastic Compute Cloud (EC2) Instances can be used to create a virtual machine. This tutorial will explain how to use a public Amazon Machine Image (AMI) that is pre-loaded with software and input data to run CMAQv5.4.

  • 1.1. Create a VM from the AWS Web Console
  • 1.2. Create a VM using the AWS Command Line
  • 1.3. Run CMAQv5.4 on c6a.2xlarge
Next
1.1. Create a VM from the AWS Web Console
Previous
Home
Copyright © 2022, CMAS Center
Made with Sphinx and @pradyunsg's Furo
Last updated on 2024-02-23 13:18:49 +0000