2. Create Single VM and run CMAQv5.3.3 (software pre-installed)

Creating an EC2 instance either from the AWS Web Interface or Command Line is easy to do. In this tutorial we will give examples on how to create and run using ec2 instances that vary in size depending on the size of the CMAQ benchmarks.

• 2.1. Use AWS Management Console to Create Single VM and run CMAQv5.3.3 (software pre-installed)
  • 2.1.1. Learn how to Use the AWS Management Console to launch EC2 instance using Public AMI
    • Public AMI contains the software and data to run 2016_12SE3 using CMAQv5.3.3
      • Login to the AWs Consol and select EC2
      • Click on the orange “Launch Instance” button
      • Enter the ami name: ami-019eb54acc4924d3f in the Search box and return or enter.
      • Note this AMI was built for the following architecture, and can be used by the c6a - hpc6a family of instances
      • Search for c6a.2xlarge Instance Type and select
      • Select key pair name or create a new key pair
      • Use the default Network Settings
      • Configure Storage
      • Select the Pull-down options for Advanced details
      • In the Summary Menu, select Launch Instance
      • Click on the link to the instance once it is successfully launched
      • Wait until the Status check has been completed and the Instance State is running
      • Click on the instance link and copy the Public IP address to your clipboard
      • Use the ssh command to login to the c6a.2xlarge instance
  • 2.1.2. Run CMAQv5.3.3 on c6a.2xlarge
    • Login to the ec2 instance
    • Login to the ec2 instance again, so that you have two windows logged into the machine.
  • 2.1.3. Load the environment modules
  • 2.1.4. Update the pcluster-cmaq repo using git
    • Verify that the input data is available
  • 2.1.5. Run CMAQv5.3.3 for 2016_12SE1 1 Day benchmark Case on 4 pe
    • Use command line to submit the job. This single virtual machine does not have a job scheduler such as slurm installed.
    • Use HTOP to view performance.
    • Successful output using the gp3 volume with hyperthreading on (8vcpus)
    • Use lscpu to confirm that there are 4 cores on the c6a.2xlarge ec2 instance that was created with hyperthreading turned off (1 thread per core).
  • 2.1.6. Save output data and run script logs
  • 2.1.7. Save the full input data, run scripts, output data and logs to an AMI that is owned by your account.
    • Go to the EC2 Dashboard
    • Click on Instances Running
    • Select Actions Pulldown menu and select Images and templates and Create Image.
    • Fill out the name of the image
    • Click Save Image
    • Click on AMI under the left menu, and then search for the image name and confirm that the status has a green checkmark and available
  • 2.1.8. Stop Instance
    • Go to the EC2 Dashboard
    • Click on Instances Running
    • Select Instance State Pulldown menu and select stop instance
    • Select Instance State Pulldown menu and select terminate instance.
    • When the pop-up menu asks if you are sure you want to terminate the instance, click on the orange Terminate button.
• 2.2. CMAQv5.3.3 on Single Virtual Machine Intermediate (software pre-installed)
  • 2.2.1. Learn how to Use AWS CLI to launch c6a.2xlarge EC2 instance using Public AMI
    • Public AMI contains the software and data to run 2016_12SE1 benchmark using CMAQv5.33
      • Verify that you can see the public AMI on the us-east-1 region.
      • AWS Resources for the aws cli method to launch ec2 instances.
    • Use the publically available AMI to launch an ondemand c6a.2xlarge ec2 instance using a io2 volume with 100000 IOPS with hyperthreading disabled
    • Obtain the code using git
      • Use the following command to obtain the public IP address of the machine.
      • Login to the ec2 instance
      • Login to the ec2 instance again, so that you have two windows logged into the machine.
    • Load the environment modules
    • Update the pcluster-cmaq repo using git
    • Run CMAQv5.4 for 12US1 Listos Training 3 Day benchmark Case on 4 pe
      • Use command line to submit the job. This single virtual machine does not have a job scheduler such as slurm installed.
      • Use HTOP to view performance.
      • Successful output
      • Change to the scripts directory
      • Use lscpu to confirm that there are 4 cores on the c6a.2xlarge ec2 instance that was created with hyperthreading turned off.
      • Edit the 12US3 Benchmark run script to use the gcc compiler and to output all species to CONC output file.
      • Run the 12US3 Benchmark case
      • Use HTOP to view performance.
      • Successful output for 12 species output in the 3-D CONC file took 56 minutes to run 1 day
      • Find the InstanceID using the following command on your local machine.
      • Stop the instance
      • Terminate Instance
      • Verify that the instance is being shut down.
  • 2.2.2. Learn how to Use AWS CLI to launch c6a.8xlarge EC2 instance using Public AMI
    • Public AMI contains the software and data to run 2016_12SE1 using CMAQv5.3.3
      • Verify that you can see the public AMI on the us-east-1 region.
      • AWS Resources for the aws cli method to launch ec2 instances.
    • Use the publically available AMI to launch an ondemand c6a.8xlarge ec2 instance using a io2 volume with 100000 IOPS with hyperthreading disabled
    • Obtain the code using git
      • Use the following command to obtain the public IP address of the machine.
      • Login to the ec2 instance (may need to wait 5 minutes for the ec2 instance to initialize and be ready for login)
      • Login to the ec2 instance again, so that you have two windows logged into the machine.
    • Load the environment modules
    • Update the pcluster-cmaq repo using git
    • Run CMAQv5.3.3 for 2016_12SE1 1 Day benchmark Case
      • Edit the run script to run on 16 cores
      • Use command line to submit the job. This single virtual machine does not have a job scheduler such as slurm installed.
      • Use HTOP to view performance.
      • Successful output
      • Find the InstanceID using the following command on your local machine.
      • Stop the instance
      • Terminate Instance
      • Verify that the instance is being shut down.
  • 2.2.3. Learn how to Use AWS CLI to launch c6a.48xlarge EC2 instance using Public AMI
    • Public AMI contains the software and data to run 2016_12SE1 using CMAQv5.3.3
      • Verify that you can see the public AMI on the us-east-1 region.
      • AWS Resources for the aws cli method to launch ec2 instances.
    • Use the publically available AMI to launch an ondemand c6a.48xlarge ec2 instance using a gp3 volume with 16000 IOPS with hyperthreading disabled
    • Obtain the code using git
      • Use the following command to obtain the public IP address of the machine.
      • Login to the ec2 instance
      • Login to the ec2 instance again, so that you have two windows logged into the machine.
    • Load the environment modules
    • Update the pcluster-cmaq repo using git
    • Run CMAQv5.3.3 for 2016_12SE1 1 Day benchmark Case on 4 pe
      • Use command line to submit the job. This single virtual machine does not have a job scheduler such as slurm installed.
      • Use HTOP to view performance.
      • Successful output
      • Use lscpu to confirm that there are 8 processors on the c6a.2xlarge ec2 instance that was created with hyperthreading turned on.
      • Run 12US2 benchmark again using gp3 volume
      • Stop the instance
      • Terminate Instance
      • Verify that the instance is being shut down.
  • 2.2.4. Documentation of Troubleshooting effort for CMAQv5.4+ on 12US1
    • Public AMI contains the software and data to run 12US1 using CMAQv5.4+
      • Verify that you can see the public AMI on the us-east-1 region.
      • AWS Resources for the aws cli method to launch ec2 instances.
    • Use a publically available AMI to launch a c6a.48xlarge ec2 instance using a gp3 volume with 16000 IOPS
      • Use the following command to obtain the public IP address of the machine.
      • Login to the ec2 instance
    • Load the environment modules
    • Run CMAQv5.4 for the 12km Listos Training Case
    • Run CMAQv5.4 for the full 12US1 Domain on c6a.48xlarge with 192 vcpus
      • Use command line to submit the job. This single virtual machine does not have a job scheduler such as slurm installed.
    • Once the model starts running (see Processing cmpleted …) in the log file, then use htop to view the CPU usage.
      • Using Cloudwatch to see the CPU utilization.
    • Save volume as a snapshot
    • Clean up Virtual Machine
      • Find the InstanceID using the following command on your local machine.
      • Terminate the instance
    • Create c6a.48xlarge with hyperthreading disabled
      • Retry the Listos run script.
      • Use HTOP to view performance.
      • Successful output
      • Used the AWS Web Interface to upgrade to an io1 system
  • 2.2.5. Use new ami instance with faster storage (io1) to create c6a.48xlarge ec2 instance
    • Create new instance
    • Check that the ec2 instance is running using the following command.
    • Use the following command to obtain the IP address
    • Login
    • Load environment modules
    • Change to the scripts directory
    • Use lscpu to confirm that there are 96 processors on the c6a.48xlarge ec2 instance that was created with hyperthreading turned off.
    • Login to the ec2 instance again, so that you have two windows logged into the machine.
    • Run 12US1 Listos Training 3 Day benchmark Case on 32 pe (this will take less than 2 minutes)
    • Successful output
    • Download input data for 12NE3 1 day Benchmark case
    • Use the aws s3 copy command to copy data from the CMAS Data Warehouse Open Data S3 bucket.
    • Link the data directory on /shared/data
    • Edit the 12US3 Benchmark run script to use the gcc compiler and to output all species to CONC output file.
    • Run the 12US3 Benchmark case
    • Successful output for 12 species output in the 3-D CONC file took 7.4 minutes to run 1 day
    • Successful output for all species output in the 3-D CONC File (222 variables)
    • Run 12US1 2 day benchmark case on 96 processors
    • Verify that it is using 99% of each of the 96 cores using htop
    • Successful run timing
    • Find the InstanceID using the following command on your local machine.
    • Stop the instance
    • Terminate Instance
    • Try creating the gp3 version of the ami using the Nitro Hypervisor, and see if that improves the performance without the cost of the io1 volume.