Building PBS Job Command Files on cherry-creek
Building PBS Job Command Files on Cherry-Creek
Building job command files is a little bit different on cherry-creek than Yucca or Eureka (due to the fact that cherry-creek is running a newer version of PBS/Pro with more capabilities). The major difference with PBS/Pro on cherry-creek is that the #PBS -l select and #PBS -l walltime statements are enforced, you must specify meaningful values for the mem= and walltime settings or your job will abort shortly after starting.
To submit a job to run on the cherry-creek cluster, a PBS job command file must be created. The job command file is a shell script that contains PBS directives; these directives are preceded by #PBS. The following is an example of a PBS command file to run a serial job, which would require only 1 processor on 1 node.
#!/bin/sh #PBS -l select=1:ncpus=1:mem=10gb #PBS -l walltime=12:00:00 #PBS -j oe #PBS -m bea #PBS -M firstname.lastname@example.org cd $PBS_O_WORKDIR your_executable commands
The first line identifies this file as a shell script. The next several lines are PBS directives that must precede any commands to be executed by the shell (e.g. the last two lines). The PBS directives are defined in the table below:
PBS Directive Function #PBS -l select=1:ncpus=1:mem=10gb Specifies a PBS resource requirement of 1 compute node and 1 processor per node. A maximum of 10gb of memory will allowed per node. #PBS -l walltime=12:00:00 Specifies a PBS resource requirement of 12 hours of wall clock time to run the job. #PBS -j oe Specifies that job output and error messages are to be joined in one file. #PBS -m bea Specifies that PBS send email notification when the job begins (b), ends (e), or aborts (a). #PBS -M email@example.com Specifies an alternate email address where PBS notification is to be sent. #PBS -V Specifies that all environment variables are to be exported to the batch job.
Here is a sample job to list names of the files contained in the users directory:
[ron@yucca ~]$ more test.pbs #!/bin/sh #PBS -l select=1:ncpus=1:mem=256mb #PBS -l walltime=12:00:00 #PBS -m bea #PBS -M firstname.lastname@example.org cd $PBS_O_WORKDIR ls -l
To submit this job, use something like:
[ron@cherry-creek ~]$ qsub test.pbs 1241.cherry-creek.nscee.edu
This will schedule the job to run on the compute queue. The system has assigned 1241 as the job number. The job's standard output and standard error will be placed in the files: test.pbs.o1241 and test.pbs.e1241 respectively.
The following is an example of a PBS email notification to the user at the end of the job:
Date: Fri, 09 May 2014 21:05:48 -0000 To: email@example.com From: firstname.lastname@example.org (root) Subject: PBS JOB 1241.yucca.nscee.edu Return-Path: <email@example.com> X-Original-To: ron@localhost Delivered-To: ron@localhost X-UIDL: O)j"!U#>"!=='!!gVR"! PBS Job Id: 1241.yucca.nscee.edu Job Name: test.pbs Execution terminated Exit_status=0 resources_used.cpupercent=0 resources_used.cput=00:00:00 resources_used.mem=0kb resources_used.ncpus=1 resources_used.vmem=0kb resources_used.walltime=00:00:01
Note that the walltime-used information in the email should be used to accurately estimate the walltime resource requirement in the PBS job command file for future job submissions so that PBS can more effectively schedule the job. When submitting a particular PBS job for the first time, the walltime requirement should be overestimated to prevent premature job termination.
After the PBS directives in the command file, the shell executes a change directory command to $PBS_O_WORKDIR, a PBS variable indicating the directory where the PBS job was submitted. Normally this will also be where the progam executable is located. Other shell commands can be executed as well. In the last line, the executable itself itself is invoked.