Requesting CPUs

To request a given number of CPUs sharing the same node, you can use the following in your SBATCH:

#SBATCH -N 1 # Number of Nodes
#SBATCH -c 5 #Number of Cores per task
or
interactive -N 1 -c 5

This will create a job with 5 CPU cores on one node.

To request a given number of CPUs spread across multiple nodes, you can use the following:

#SBATCH -N 2-4    # number of nodes to allow tasks to spread across (MIN & MAX)
#SBATCH -n 10    # number of TASKS
#SBATCH -c 5     # CPUs per TASK
or
interactive -N 2-4 -n 10 -c 5

The above example will allocate a total of 50 cores spread across as few as 2 nodes or as many as 4 nodes.

Take note of the inclusion or omission of -N:

#SBATCH -c 5     # CPUs per TASK
#SBATCH -n 10    # number of TASKS
or
interactive -n 10 -c 5

This reduced example will still allocate 50 cores, 5 cores per task on any number of available nodes. Note, that unless you are using MPI-aware software, you will likely prefer to always add -N, to ensure that each job worker has sufficient connectivity.

-c and -n have similar effects in Slurm in allocating cores, but -n is the number of tasks, and -c is the number of cores per task. MPI processes bind to a task, so the general rule of thumb is for MPI jobs to allocate tasks, while serial jobs allocate cores, and hybrid jobs allocate both.

See the official Slurm documentation for more information: https://slurm.schedmd.com/sbatch.html

Requesting Memory

Cores and memory are de-coupled: if you need only a single CPU core but ample memory, you can do so like this:

#SBATCH -c 1
#SBATCH -N 1
#SBATCH --mem=120G
or
interactive -N 1 -c 1 --mem=120G

If you do not specify --mem, you will be allocated 2GiB per CPU core OR 24GiB per GPU

To request more than 512GiB of memory, you will need to use the highmem partition

#SBATCH -p highmem
#SBATCH --mem=1400G

To request all available memory on a node:

#SBATCH --exclusive 
#SBATCH --mem=0

Requesting GPUs

To request a GPU, you can specify the -G option within your job request.

This will allocate the first available GPU that fits your job request:

#SBATCH -G 1
or 
interactive -G 1

To request multiple GPUs specify a number greater than 1:

#SBATCH -G 4
or 
interactive -G 4

To request a specific number of GPUs per node when running multi-node:

#SBATCH -N 2              # Request two nodes
#SBATCH --gpus-per-node=2 #Four total GPUs, two per node

To request a specific type of GPU (a100 for example):

#SBATCH -G a100:1
or
interactive -G a100:1

GPU Varieties Available

Below is a table demonstrating the available GPU instance sizes you can allocate:

The a100s can come in two varieties, as seen above.

To guarantee a 80GB a100, include this feature: #SBATCH -C a100_80. This can be done also with interactive -C a100_80 (a100_40 is also provided). To request more than one a100s while specifying the variety:

$ interactive -G a100:2 -C a100_80
or
#SBATCH -G a100:2
#SBATCH -C a100_80