Job Arrays

Job arrays allow you to submit many similar jobs efficiently without creating separate scripts for each task. Each array task gets a unique ID that you can use to process different inputs or parameters.

Basic Array Syntax

#!/bin/bash
#SBATCH --job-name=array-job
#SBATCH --array=1-100
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=2
#SBATCH --mem=4G
#SBATCH --time=01:00:00

# $SLURM_ARRAY_TASK_ID ranges from 1 to 100
python process_sample.py --id $SLURM_ARRAY_TASK_ID

Key Environment Variables

  • $SLURM_ARRAY_TASK_ID — The unique ID for this array task (ranges from your specified bounds)

  • $SLURM_ARRAY_JOB_ID — The master job ID for the entire array (same as $SLURM_JOB_ID)

Limiting Concurrent Tasks

Control how many array tasks run simultaneously using the %N syntax:

#SBATCH --array=1-100%10

This runs 100 tasks total, but only 10 at a time. The remaining 90 wait in the queue. This is useful for:

  • Limiting resource consumption

  • Managing I/O load on shared filesystems

  • Staggering job execution to avoid queue congestion

Unique Output Files

Each array task can write to a separate output file using special placeholders:

#SBATCH --output=output_%A_%a.txt
#SBATCH --error=error_%A_%a.txt

  • %A — Master job ID

  • %a — Array task ID

Example: Task 5 would create output_12345_5.txt and error_12345_5.txt.

Common Array Patterns

Processing Multiple Files

Process a list of files efficiently:

#!/bin/bash
#SBATCH --job-name=process_files
#SBATCH --array=1-100
#SBATCH --time=01:00:00

# Get the Nth file from the list
FILE=$(ls data/*.txt | sed -n "${SLURM_ARRAY_TASK_ID}p")
./process_script.sh "$FILE"

Parameter Sweeps

Run simulations across a range of parameters:

#!/bin/bash
#SBATCH --job-name=param_sweep
#SBATCH --array=0-9
#SBATCH --time=00:30:00

# Calculate parameter value from task ID
PARAM=$((SLURM_ARRAY_TASK_ID * 10))
./run_simulation.sh --param "$PARAM"

This creates 10 tasks with parameters: 0, 10, 20, 30, …, 90.

Custom Task Ranges

You can specify non-contiguous task IDs:

# Specific task IDs
#SBATCH --array=5,10,15,20

# Multiple ranges
#SBATCH --array=1-50,100-150

# Skip tasks (1-100, then 200-300)
#SBATCH --array=1-100,200-300

Large-Scale Arrays

For very large numbers of tasks:

#SBATCH --array=1-10000%50   # 10,000 tasks, 50 concurrent

Best Practices

  1. Use unique output files — Always include %A_%a in --output and --error to avoid overwriting

  2. Limit concurrency — Use %N to prevent overwhelming shared resources

  3. Check task bounds — Ensure your script handles the full range of task IDs

  4. Test small first — Submit a small array (e.g., --array=1-5) before scaling up

  5. Handle missing inputs — Add error checking if some files might not exist

Example: Batch Image Processing

#!/bin/bash
#SBATCH --job-name=image-process
#SBATCH --array=1-500%20
#SBATCH --cpus-per-task=4
#SBATCH --mem=8G
#SBATCH --time=02:00:00
#SBATCH --output=logs/%A_%a.log

IMAGE=$(ls images/*.jpg | sed -n "${SLURM_ARRAY_TASK_ID}p")
echo "Processing $IMAGE (task $SLURM_ARRAY_TASK_ID)"
convert "$IMAGE" -resize 50% "processed/$(basename $IMAGE)"

This processes 500 images with only 20 running at once, logging each task separately.