Skip to content

Building containers

Apptainer containers are built from a definition file which allows the container to be built almost identically by anyone possessing the file.

Use a compute node

Building containers must be done on a compute node. Please request an interactive salloc session with an appropriate resource request, remembering to follow the APPTAINERENV_SLURM_NTASKS=${SLURM_NTASKS} advice covered below to ensure all cores are used correctly. Multiple cores and plenty of RAM are highly recommended, particularly for larger containers.

Warning regarding the use of ${SLURM_NTASKS} variable

When building containers using multiple cores, you should run the following command to use the ${SLURM_NTASKS} variable within the container: APPTAINERENV_SLURM_NTASKS=${SLURM_NTASKS} apptainer build ... or export this variable before building the container. Failure to set this may result in memory errors, or poor threading performance when attempting to use this variable, as the build shell would not know how many cores to use. For example, make -j ${SLURM_NTASKS} would become make -j, which attempts to run make with all CPU cores on the node, rather than the number requested.

Root privileges are not required to build a container

Apptainer does not require root access to build containers on Apocrita. Users are therefore encouraged to build containers in their home, scratch or lab directories as required, within a cluster job.

Alternatively, you can:

  • create the container on your own machine and copy it to Apocrita
  • create the container inside a virtual machine
  • submit a container definition file for provisioning by the ITSR support team
  • pull pre-built external images

One primary task per container

HPC containers are designed to perform one primary task, and should consist of a main application and its dependencies, in a similar way to how module files are provided. Since containers are lightweight, you can use separate containers instead of general purpose containers containing a collection of applications. This improves supportability, performance and reproducibility.

Building containers for other Linux distributions

You can create containers from an existing Apptainer or Docker image, as explained in the following section.

Using LTS for Ubuntu definitions

When building an Ubuntu container we recommend that you use a release with long term support (LTS release). Non-LTS Ubuntu releases have very limited support cycles which may lead to difficulties downloading packages if used after their end-of-life date.

Building containers from an existing local base image

This enables you to either build or use an existing container as a base image to build other containers. Base images must be built first if part of a dependency chain and are no longer required once all dependent containers have been built.

Apptainer local images

The following example demonstrates the creation of a local base Ubuntu 24.04 image using the definition file ubuntu24_base.def, and then creating another container with python3 installed, using the local base image:

Bootstrap: docker
From: ubuntu:24.04

Create the base image:

apptainer build ubuntu24_base.sif ubuntu24_base.def

The non-base image container (i.e. python3 in this example) can be built using the definition file ubuntu24_python3.def:

Bootstrap: localimage
From: ubuntu24_base.sif

%post
  apt-get update
  apt-get install --yes python3

%runscript
  python3 "${@}"

To build from the base image:

apptainer build ubuntu24_python3.sif ubuntu24_python3.def

Docker images

You can bootstrap from Docker containers, although if supplied by a third party, you have less visibility or control over these images, so use with caution, as this may impact the future reproducibility of results.

The below example demonstrates installing the python3 package within an Ubuntu 24.04 container using the definition file ubuntu24_docker_python3.def, which imports the ubuntu:24.04 base container available on the Docker Hub:

Bootstrap: docker
From: ubuntu:24.04

%post
  apt-get update
  apt-get install --yes python3

%runscript
  python3 "${@}"

Build the container. This will produce a container similar to the previous examples, but may vary slightly in overall size depending on packages installed in the base Docker image:

apptainer build ubuntu24_docker_python3.sif ubuntu24_docker_python3.def

Future-proofing your containers

When building your own containers, be sure to make them portable and future-proof.

  • Consider whether the container will still build and produce the same results if the OS release or application version changes.
  • If copying files from a working directory as part of setup is unavoidable, ensure that any files copied from the working directory are are available for others to download (i.e. in a Git repository if not large).
  • Perform all setup as part of the build process. If any manual steps are performed after the container is built, they should be integrated within the definition file, and the container rebuilt.
  • Consider if the ability to rebuild your container will be impacted by package updates, or deprecation of old releases.

Definition file sections

The following example definition file demonstrates commonly used definition file sections:

  • %help
  • %post
  • %environment
  • %test
  • %runscript

Help section

The %help section is designed to provide information about the container when apptainer run-help is run on the container, for example:

$ apptainer run-help /share/apps/rocky9/containers/public/python3_helloworld.sif
Purpose: Test container to print "Hello, World!" in Python3.
Author:  ITS Research / QMUL.

Post section

The %post section contains the commands used to build the container, such as package installs, file downloads, compilation and software configuration.

Environment section

Environment settings supplied at build-time in the %post section are only set during build-time and are not available at run-time. Environment settings which need to be available at run-time should be added to the %environment section.

Test section

The %test section defines a set of commands or tests which should be run to validate the container has been built successfully. Some example tests include:

  • installed binaries are available on the $PATH variable
  • --help or --version parameter for binaries (if supported)
  • libraries, header files and man pages exist

All tests will be run during the build process, after %post has completed. To build a container without running the tests, pass the -T or --notest option to the apptainer build command.

To run the tests for an existing container, run the apptainer test command, for example:

$ apptainer test /share/apps/rocky9/containers/public/python3_helloworld.sif
/usr/bin/python3

Runscript section

The %runscript section defines the default action a container will perform when run as an executable or using apptainer run. This is configured during the build process.

Application parameters or arguments

If the runscript calls an application which takes parameters or arguments, include "${@}" after the application otherwise anything passed after the container name will be ignored by Apptainer.

Inspecting a container

To display information about how a container was build, use the apptainer inspect command. The -d option for this command will print the definition file used to built the container and the -r option will print the runscript (if added during build-time). For example:

$ apptainer inspect -d /share/apps/rocky9/containers/public/python3_helloworld.sif
Bootstrap: docker
From: ubuntu:24.04

%help
  Purpose: Test container to print "Hello, World!" in Python3.
  Author:  ITS Research / QMUL.

%post
  apt-get update
  apt-get install --yes python3

  apt-get clean && \
   rm -rf /var/lib/apt/lists/*

%test
  which python3

%runscript
  python3 -c 'print("Hello, World!")'

The apptainer help inspect command provides additional options for inspecting the container.