2. Create a Parallel Cluster and run CMAQv5.4

Why might I need to use ParallelCluster?

The AWS ParallelCluster may be configured to be the equivalent of a High Performance Computing (HPC) environment, including using job schedulers such as Slurm, running on multiple nodes using code compiled with Message Passing Interface (MPI), and reading and writing output to a high performance, low latency shared disk. The advantage of using the AWS ParallelCluster command line interface is that the compute nodes can be easily scaled up or down to match the compute requirements of a given simulation. HPC compute nodes such as hpc6a or hpc7g are available in a limited set of regions at significantly discounted pricing (60% below on demand costs). Users can also attempt to reduce costs by using Spot instances rather than On-Demand for the compute nodes. ParallelCluster also supports submitting multiple jobs to the job submission queue.

Our goal is make this user guide to running CMAQ on a ParallelCluster as helpful and user-friendly as possible. Any feedback is both welcome and appreciated.

• 2.1. Build a Demo ParallelCluster
  • 2.1.1. Install Parallel Cluster AWS CLI 3.0
  • 2.1.2. Configure a Demo Cluster
  • 2.1.3. Create a Demo Cluster
  • 2.1.4. Login to the ParallelCluster
  • 2.1.5. Exit the cluster
  • 2.1.6. Delete the Demo Cluster
• 2.2. Use ParallelCluster with Software and Data pre-installed on hpc7g.16xlarge
  • 2.2.1. Configure the ParallelCluster
  • 2.2.2. Create the hpc7g pcluster
• 2.3. Run CMAQ on hpc7g.16xlarge
  • 2.3.1. Login to cluster
  • 2.3.2. Preloading the files
  • 2.3.3. Run the 12US1 Domain on 128 cores
  • 2.3.4. Submit a job to run on 192 pes, 3x64 nodes
  • 2.3.5. Submit a job to run on 320 pes running on 5 nodes
  • 2.3.6. Submit a job to run on 128 cores with 32 cores per node.
• 2.4. Run DESID CMAQ on hpc7g.16xlarge
  • 2.4.1. Run CMAQ for DESID
    • Edit the DESID Namelist
    • Edit runscript to use DESID Namelist
    • Run CMAQ using DESID
    • Review Log file from DESID run
• 2.5. Use ParallelCluster with Software and Data pre-installed for c6a.48xlarge
  • 2.5.1. Create CMAQ Cluster using SPOT pricing
  • 2.5.2. Create the c6a.48xlarge pcluster
• 2.6. Run CMAQ on c6a.48xlarge
  • 2.6.1. Login to cluster
  • 2.6.2. Verify that the input data is imported to /fsx from the S3 Bucket
  • 2.6.3. Run the 12US1 Domain on 192 pes
  • 2.6.4. Submit a request for a 96 pe job ( 1 x 96 pe) or 1 nodes instead of 2 nodes
  • 2.6.5. Submit a job to run on 288 pes, 3x96 nodes