STAR-CCM+

STAR-CCM+ is a complete multi-disciplinary platform for the simulation of products and designs operating under real-world conditions.

STAR-CCM+ is available as a module on Apocrita.

Usage

To run the default version of STAR-CCM+, simply load the starccm module:

module load starccm

For usage documentation, run starccm+ -help

Example jobs

When running jobs, replace SIM-FILE in the examples below with the name of your simulation file.

Serial job

Here is an example job running on 4 cores and 16G of memory on a single node:

#!/bin/bash
#SBATCH -n 4
#SBATCH --time=1:0:0
#SBATCH --mem-per-cpu=4G

module load starccm

starccm+ -batchsystem slurm \
  -batch SIM-FILE

Parallel job

Here is an example job running on 96 cores across 2 ddy nodes with MPI:

#!/bin/bash
#SBATCH -N 2
#SBATCH -n 96
#SBATCH -p parallel
#SBATCH -t 240:0:0
#SBATCH --exclusive
#SBATCH --mem=0

module load starccm

starccm+ -batchsystem slurm \
  -mpi intel \
  -batch SIM-FILE

GPU job

Do not use NVIDIA MPS

STAR-CCM+ integrates the NVIDIA Multi-Process Service (MPS) but this should not be used on Apocrita as it interferes with correct GPU allocation and prevents other users on the same node from using their assigned GPU(s). Please ensure you add -gpgpu auto:nomps to all GPU jobs to ensure that MPS is disabled (see below example).

Here is an example job running on 8 cores and 1 GPU:

#!/bin/bash
#SBATCH -n 8
#SBATCH -p gpu
#SBATCH --time=1:0:0
#SBATCH --mem-per-cpu=11G
#SBATCH --cpus-per-gpu=8
#SBATCH --gres=gpu:1

module load starccm

starccm+ -batchsystem slurm \
  -mpi intel \
  -batch SIM-FILE \
  -gpgpu auto:nomps