Submitting Jobs / Slurm Scripts

The most simple way of submitting a job to Slurm is by creating a script for your job, then submitting that script by using the program sbatch.

For example, let's say I have the following script, named myjob.sh, which simply prints out the text "Hello, world!":

#!/bin/bash
echo "Hello, world!"

I could then submit this to Slurm by running sbatch myjob.sh. Slurm will give us the ID of the job, for example:

Submitted batch job 8985982

Slurm will receive the script, determine which of the computing nodes are best suited for runnning this script, send the script to that node and run it, then when the program is finished running, it will output a file in the format slurm-<ID>.out with whatever the program outputs. In this case, I should see a file named slurm-8985982.out with the contents Hello, world!.

In this case, we provide no configuration options to sbatch, so it will submit the job with default options. However, sbatch has options for specifying things like the number of nodes to submit a job to, how many CPU cores to use, and who to email regarding the status of a job. These options are useful, so when submitting a job, it's worth specifying them. sbatch allows us to specify these options in a script by providing comments at the beginning of the script. For example:

#!/bin/bash
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --mail-type=END	
#SBATCH [email protected]
###################################
echo print all system information
uname -a
echo print effective userid
whoami
echo Today date is:
date
echo Your current directory is:
pwd
echo The files in your current directory are:
ls -lt
echo Have a nice day!
sleep 20

• SBATCH -N 1 - Specifies the number of nodes Slurm should submit the job to. You must always set -N 1 unless you receive special instruction from the system admin.

• SBATCH -n 1 - Specifies the number of cores Slurm should delegate to the job. Don't specify more than 16 unless approved by the system admin.

• SBATCH --mail-type=END - Specifies when you should receive an email notification regarding the job. Options are BEGIN,END,FAIL,ALL.

• SBATCH [email protected] - Specifies the email that should receive notifications about the status of this job.

Submitting Jobs to Specific Nodes

To submit your job to a specific node, use the following command:

sbatch -w [cX] [script_file]

Where X is a number specifying the node you intend to use. For example, the following command will submit myjob.sh to node c5:

sbatch -w c5 myjob.sh

To submit your job while excluding nodes (for example exclude nodes c5 to c22, use the following command:

sbatch --exclude c[5-22] myjob.sh

The same flags are applicable when running srun .

You can also add these flags to your script as SBATCH comments instead of submitting them as command line flags. For example, to submit a script to sbatch which you'd like to be submitted to node c5, you'd write:

#!/bin/bash
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --mail-type=END	
#SBATCH [email protected]
#SBATCH -w c5
###################################
echo print all system information
uname -a
echo print effective userid
whoami
echo Today date is:
date
echo Your current directory is:
pwd
echo The files in your current directory are:
ls -lt
echo Have a nice day!
sleep 20

Submitting Jobs to Specific Partitions

To submit your jobs to a specific partition, use the following command:

sbatch -p [partition name]

Where [partition name] is one of: normal, bcc, kellis . If no partition is provided, Slurm defaults to normal.

You can also add this flag to your script as SBATCH comments instead of submitting them as command line flags. For example, to submit a script to sbatch which you'd like to be submitted to node c5, you'd write:

#!/bin/bash
#SBATCH -N 1
#SBATCH -n 1
#SBATCH --mail-type=END	
#SBATCH [email protected]
#SBATCH -p bcc
###################################
echo print all system information
uname -a
echo print effective userid
whoami
echo Today date is:
date
echo Your current directory is:
pwd
echo The files in your current directory are:
ls -lt
echo Have a nice day!
sleep 20

The same flag is applicable when running srun .

Monitoring and Controlling a Job

To monitor the progress of your job use the command squeue. To display information relative only to the jobs you submitted, use the following (where username is your username):

squeue -u username

Cancelling a Job

To cancel a job that you are running, your can use the scancelcommand and pass it your job ID:

scancel <JOB ID>

scancel 1234567