2.1.6. Save output data and run script logs#

Copy the log files and the output data to an s3 bucket.

cd /shared/pcluster-cmaq/s3_scripts

cat s3_upload_cmaqv533.c6a.2xlarge.csh

Output

#!/bin/csh -f
# Script to upload output data to S3 bucket
# need to set up your AWS credentials prior to running this script
# aws configure
# NOTE: need permission to create a bucket and write to an s3 bucket. 
# 

mkdir /shared/data/output/logs
mkdir /shared/data/output/scripts

cp /shared/build/openmpi_gcc/CMAQ_v533/CCTM/scripts/*.log /shared/data/output_CCTM_v533_gcc_Bench_2016_12SE1/logs/
cp  /shared/build/openmpi_gcc/CMAQ_v533/CCTM/scripts/run_cctm_Bench_2016_12SE1.csh /shared/data/output_CCTM_v533_gcc_Bench_2016_12SE1/scripts/

setenv BUCKET c6a.2xlarge.cmaqv533
aws s3 mb s3://$BUCKET
aws s3 cp --recursive /shared/data/output_CCTM_v533_gcc_Bench_2016_12SE1 s3://$BUCKET

Set your aws credentials by running the command

aws configure

Edit the script to create a unique bucket name

Run the script

./s3_upload_cmaqv533.c6a.2xlarge.csh

or

2.1.7. Save the full input data, run scripts, output data and logs to an AMI that is owned by your account.#

Go to the EC2 Dashboard#

EC2 Resources on AWS Web Console

Click on Instances Running#

Select the checkbox next to the c6a.2xlarge instance name

Select Instance on EC2 Dashboard

Select Actions Pulldown menu and select Images and templates and Create Image.#

Note, this will log you out of the ec2 instance, and should be done after all runs have been completed and your are ready to save the image.

Create Image on EC2 Dashboard

Fill out the name of the image#

Name the instance to help identify the ec2 instance type, CMAQ version installed, and perhaps the input/output data available

Confirm Save Image on EC2 Dashboard

Click Save Image#

Wait until the image status available before terminating the ec2 instance

Click on AMI under the left menu, and then search for the image name and confirm that the status has a green checkmark and available#