GPU Profiling¶

Learning Objectives

Use Nsight Systems to profile the full application timeline (CPU + GPU)
Use Nsight Compute to analyze individual CUDA kernels
Identify kernel bottlenecks: occupancy, memory throughput, compute utilization
Compare kernel performance to GPU theoretical limits
Apply optimizations based on profiling data

GPU Profiling Overview¶

CPU profilers (LIKWID, Valgrind) cannot see inside GPU kernels. For CUDA, NVIDIA provides the Nsight suite:

Tool	Purpose
Nsight Systems (`nsys`)	Full application timeline: CPU threads, GPU kernels, memory transfers
Nsight Compute (`ncu`)	Per-kernel analysis: occupancy, memory throughput, SM utilization

Nsight Systems — Timeline Profiling¶

Captures the full execution timeline:

nsys profile -o output ./program
nsys stats output.qdrep

What You’ll See¶

GPU kernel launch times and durations
Memory transfers (H2D, D2H) — often the bottleneck
CPU-GPU overlap (or lack thereof)
Stream synchronization points

Identifying Bottlenecks¶

Look for:

Long memory transfers: If cudaMemcpy takes longer than the kernel, consider async transfers or unified memory.
Kernel gaps: If the GPU sits idle between kernels, launch them in streams for overlap.
CPU bottlenecks: If the CPU is the limiting factor, optimize host code or use async APIs.

Nsight Compute — Kernel Analysis¶

Deep dive into individual kernels:

ncu --section SpeedOfLight --section Occupancy ./program

Key Metrics¶

Metric	What It Tells You
SM Active Utilization	% of time SMs are doing useful work
L1/Shared Memory Throughput	Memory subsystem efficiency
Global Memory Throughput	DRAM bandwidth utilization
Occupancy	% of theoretical warps active
Speed of Light	How close to theoretical limits

Example Output¶

Speed of Light Analysis:
  Achieved: 1456 GB/s
  Theoretical: 2000 GB/s
  Efficiency: 72.8%

Occupancy:
  Active Warps: 1024
  Max Warps: 2048
  Occupancy: 50%

Common Optimization Patterns¶

1. Memory Coalescing¶

Problem: Uncoalesced accesses (random strides) waste bandwidth.

Fix: Restructure data layout (SoA instead of AoS) or access patterns.

2. Shared Memory Tiling¶

Problem: Repeated global memory reads of the same data.

Fix: Load tiles into shared memory for reuse.

3. Increasing Occupancy¶

Problem: Low occupancy limits latency hiding.

Fix: Reduce register usage, increase block size, or simplify control flow.

Running on ARC Clusters¶

Nsight tools are available on GPU nodes:

module load cuda

# Profile with Nsight Systems
nsys profile -o my_profile ./cuda_program

# Profile with Nsight Compute
ncu --launch-skip 0 --launch-count 1 ./cuda_program

Note

Check cluster policy — some clusters require interactive jobs for profiling tools.