Contents Menu Expand Light mode Dark mode Auto light/dark mode
aws-cmaq documentation
Logo
aws-cmaq documentation

Contents:

  • 1. Create Single VM
    • 1.1. Create a VM from the AWS Web Console
    • 1.2. Create a VM using the AWS Command Line
    • 1.3. Run CMAQv5.4 on c6a.2xlarge
  • 2. Create a Parallel Cluster and run CMAQv5.4
    • 2.1. Build a Demo ParallelCluster
    • 2.2. Use ParallelCluster with Software and Data pre-installed on hpc7g.16xlarge
    • 2.3. Run CMAQ on hpc7g.16xlarge
    • 2.4. Run DESID CMAQ on hpc7g.16xlarge
    • 2.5. Use ParallelCluster with Software and Data pre-installed for c6a.48xlarge
    • 2.6. Run CMAQ on c6a.48xlarge
  • 3. Performance and Cost Optimization
    • 3.1. ParallelCluster Configuration
    • 3.2. CMAQv5.4 Benchmarks
    • 3.3. Slurm Compute Node Provisioning
    • 3.4. Benchmark Timings for CMAQv5.4 12US1 Benchmark
    • 3.5. Benchmark Scaling Plots for CMAQv5.4 12US1 Benchmark
    • 3.6. Cost Information
    • 3.7. Recommended Workflow for extending to annual run
  • 4. Developer Guide to install and run CMAQv5.4 on Single VM or Parallel Cluster
    • 4.1. Install CMAQv5.4+ on Single Virtual Machine Advanced (optional)
      • 4.1.1. Install Software and run CMAQv5.4 on c6a.xlarge for the 12km Listos Training Domain
      • 4.1.2. Install Software and run CMAQv5.4 on c7g-hpc7g for the 12km Listos Training Domain
      • 4.1.3. Install I/O API libraries that support HDF5
      • 4.1.4. Upgrade to run CMAQ on larger EC2 Instance
    • 4.2. Install CMAQv5.4 on ParallelCluster (optional)
      • 4.2.1. Configure Parallel Cluster
      • 4.2.2. Create the hpc7g.16xlarge pcluster
      • 4.2.3. Install CMAQ sofware and libraries on ParallelCluster version 3.6
      • 4.2.4. Install netCDF libraries that use HDF5 and support nc4 compressed files
      • 4.2.5. Install gh following these instructions
      • 4.2.6. Use gh authentication
      • 4.2.7. Run CMAQ using hpc7g.16xlarge compute nodes
      • 4.2.8. Run CMAQ using hpc7g.8xlarge compute nodes
      • 4.2.9. Install Input Data on ParallelCluster
  • 5. Post-process and QA
    • 5.1. Post-process CMAQ and Install R
      • 5.1.1. Scripts to run combine and post processing
      • 5.1.2. Install R, Rscripts and Packages
    • 5.2. QA CMAQ
      • 5.2.1. Quality Assurance
      • 5.2.2. Run m3diff to compare the output data for two runs that have different values for NPCOL
      • 5.2.3. Run an R script to create the box plots and spatial plots comparing the output of two runs
    • 5.3. Compare Timing of CMAQ Routines
      • 5.3.1. Parse timings from the log file
    • 5.4. Copy Output to S3 Bucket
      • 5.4.1. Copy Output Data and Run script logs to S3 Bucket
      • 5.4.2. Copy scripts and logs to /fsx
      • 5.4.3. Examine the output files
      • 5.4.4. Copy the output to an S3 Bucket
  • 6. Logout and Delete ParallelCluster
    • 6.1. Logout of cluster when you are done
    • 6.2. Delete Cluster
    • 6.3. Verify that the cluster was deleted
  • 7. Additional Resources
    • 7.1. FAQ
    • 7.2. Free Training
    • 7.3. Another workshop to learn the AWS CLI 3.0
    • 7.4. Youtube video
    • 7.5. Intro to AWS for HPC People - HPC Tech Shorts
    • 7.6. Benchmarking
    • 7.7. Help Resources for CMAQ
    • 7.8. Computing on the Cloud References
    • 7.9. AWS Resources for the aws cli method to launch ec2 instances.
    • 7.10. Resources from AWS for diagnosing issues with running the Parallel Cluster
    • 7.11. Instructions on how to create Parallel Cluster Amazon Machine Image (AMI) from the command line
    • 7.12. ParallelCluster Update
    • 7.13. Use Elastic Fabric Adapter/Elastic Network Adapter for better performance
    • 7.14. VPC Management
    • 7.15. Using Cost Allocation Tags with ParallelCluster
  • 8. Future Work
    • 8.1. Future Work
  • 9. Contribute to this Tutorial
    • 9.1. Contribute to Pcluster-cmaq Documentation
Back to top
Edit this page

7.11. Instructions on how to create Parallel Cluster Amazon Machine Image (AMI) from the command line#

Tutorial How-to Create AMI from Command Line

We also need to have additional protections if we make these AMI’s public.

Building Shared AMIs

Securing Access to AMIs for AWS Marketplace

Building Pcluster from Existing AMI

Next
7.12. ParallelCluster Update
Previous
7.10. Resources from AWS for diagnosing issues with running the Parallel Cluster
Copyright © 2022, CMAS Center
Made with Sphinx and @pradyunsg's Furo
Last updated on 2024-02-23 13:18:49 +0000