Requesting GPU Resources

Basic GPU Request

The key directive for GPU jobs is --gres (generic resources):

#SBATCH --gres=gpu:1        # Request 1 GPU
#SBATCH --gres=gpu:2        # Request 2 GPUs
#SBATCH --gres=gpu:a100:1   # Request 1 A100 specifically

Complete Example

#!/bin/bash
#SBATCH --job-name=gpu-job
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=4
#SBATCH --gres=gpu:1
#SBATCH --mem=32G
#SBATCH --time=02:00:00
#SBATCH --partition=<valid_gpu_partition>
#SBATCH --output=gpu_job.out

# Load required modules
module purge
module load cuda cudnn

# Verify GPU availability
nvidia-smi

# Run your application
python train_model.py

GPU Types by Cluster

A100 GPUs only with MIG support.

#SBATCH --partition=gpu-a100
#SBATCH --gres=gpu:a100:1

MIG Support: A100 GPUs support Multi-Instance GPU (MIG) - splitting one GPU into multiple instances:

# Request 1 MIG instance (7 instances per GPU)
#SBATCH --partition=gpu-a100-mig7
#SBATCH --gres=gpu:mig:1g.10gb

# Request 2 MIG instances
#SBATCH --partition=gpu-a100-mig2
#SBATCH --gres=gpu:mig:2g.20gb

Check available MIG configurations with nvidia-smi -q on a GPU node.

Mixed GPU types (V100, A100, L40S).

# V100 GPUs
#SBATCH --partition=gpu-v100
#SBATCH --gres=gpu:v100:1

# A100 GPUs
#SBATCH --partition=gpu-a100
#SBATCH --gres=gpu:a100:1

# L40S GPUs
#SBATCH --partition=gpu-l40s
#SBATCH --gres=gpu:l40s:1

A100 GPUs only.

#SBATCH --partition=gpu-a100
#SBATCH --gres=gpu:a100:1

GPU Memory Considerations

GPU Type

Memory

Best For

Available On

V100

32 GB

Medium models, inference

Atlas

A100

80 GB

Large models, training

Ptolemy, Atlas, Morrill

L40S

48 GB

Inference, graphics

Atlas

Multi-GPU Jobs

For jobs using multiple GPUs:

#SBATCH --gres=gpu:4
#SBATCH --cpus-per-task=16  # 4 CPUs per GPU recommended

# Your application should handle multi-GPU
python train_multi_gpu.py

Tips for Selecting a Cluster

Best for:

  • Large model training (80GB A100 memory)

  • MIG-based multi-tenant workloads

  • Consistent GPU environment

Considerations:

  • Single GPU type (A100 only)

  • Smallest GPU cluster

Best for:

  • Mixed workload experimentation

  • V100 legacy code support

  • L40S inference workloads

  • Largest GPU capacity

Considerations:

  • Multiple GPU types (specify if needed)

  • Most partitions to choose from

Best for:

  • Large-scale GPU training

  • A100-specific workloads

  • High-memory GPU jobs

Considerations:

  • A100 only

  • Large-scale jobs preferred

Checking GPU Availability

# Check which GPUs are available
showpartitions -g

# Check GPU status on a node
scontrol show nodes <node-name>