Working with containers

What is Docker?

Standardized packaging for software, dependencies as well as the operating system
Docker lets you create and run applications securely isolated in containers

Docker in life science

Docker can also be used to define software environments and settings for benchmarking studies:

CAMI Challenge: an independent evaluation of several tools in the field of metagenomics.

[…] We defined standards for submitting the software itself, along with parameter settings and required databases and implemented them in Docker container templates … ¹

Docker in life science

Docker can also be used to define software environments and settings for benchmarking studies:

2nd CAMI Challenge

For reproducibility, participants could submit a Docker container containing the complete workflow, a Bioconda script or a software repository with detailed installation instructions […] ¹

Docker nomenclature

A Docker image is a stand-alone executable package of software
A Docker container is an instance of a Docker image
A Dockerfile is a recipe used to build a Docker image
Docker Hub is an online service for hosting Docker images

An example: using a different OS

Check OS on local machine

$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64

Pull Ubuntu (Linux) image

$ docker pull ubuntu:16.04
16.04: Pulling from library/ubuntu
22dc81ace0ea: Pull complete
...
Digest: sha256:e348fbbea0e0a0e73ab0370de151e7800684445c509d46195aef73e090a49bd6
Status: Downloaded image for ubuntu:16.04$ docker pull ubuntu:16.04
16.04: Pulling from library/ubuntu
22dc81ace0ea: Pull complete
...
Digest: sha256:e348fbbea0e0a0e73ab0370de151e7800684445c509d46195aef73e090a49bd6
Status: Downloaded image for ubuntu:16.04$ docker pull ubuntu:16.04
16.04: Pulling from library/ubuntu
22dc81ace0ea: Pull complete
...
Digest: sha256:e348fbbea0e0a0e73ab0370de151e7800684445c509d46195aef73e090a49bd6
Status: Downloaded image for ubuntu:16.04

An example: using a different OS

Check OS on local machine

$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64

Run the container interactively and check OS version

$ docker run -it ubuntu:16.04
root@407b0fd13fe5:/## uname -a
Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux$ docker run -it ubuntu:16.04
root@407b0fd13fe5:/## uname -a
Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux$ docker run -it ubuntu:16.04
root@407b0fd13fe5:/## uname -a
Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux$ docker run -it ubuntu:16.04
root@407b0fd13fe5:/## uname -a
Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux

Mounting volumes

Local project/ directory:

$ ls
|- doc/
|- data/
|- code/
|- logs/
|- results/
|- Snakefile
|- config.yml
|- environment.yml
|- Dockerfile
|- README.md$ ls
|- doc/
|- data/
|- code/
|- logs/
|- results/
|- Snakefile
|- config.yml
|- environment.yml
|- Dockerfile
|- README.md$ ls
|- doc/
|- data/
|- code/
|- logs/
|- results/
|- Snakefile
|- config.yml
|- environment.yml
|- Dockerfile
|- README.md

Docker container file system:

$ docker run \
    -v $PWD/data:/home/data \
    ubuntu:16.04 \
    ls

|- bin/
|- dev/
|- etc/
|- home/
|  |- data/
|- lib/
|- sys/
|- usr/$ docker run \
    -v $PWD/data:/home/data \
    ubuntu:16.04 \
    ls

|- bin/
|- dev/
|- etc/
|- home/
|  |- data/
|- lib/
|- sys/
|- usr/$ docker run \
    -v $PWD/data:/home/data \
    ubuntu:16.04 \
    ls

|- bin/
|- dev/
|- etc/
|- home/
|  |- data/
|- lib/
|- sys/
|- usr/

What can I use Docker for?

As an advanced environment manager

docker run -it -v $PWD:/home my_image /bin/bash

To package your code with the environment it needs

docker run \
    -v $PWD/data:/home/data \
    -v $PWD/results:/home/results \
    my_image snakemake report.pdf

To package a whole project with environment, code and data (e.g. to accompany a manuscript).

docker run \
    -v $PWD/results:/home/results \
    my_image snakemake report.pdf

What is Apptainer?

Another software for working with containers, similar to Docker
An apptainer image is contained in a single file, facilitating portability
Does not require root access, making it suitable for work at HPCs

What is Apptainer?

Apptainer used to be ‘Singularity’.
In 2021 Singularity joined the Linux Foundation and became Apptainer.
The company Sylabs still maintains Singularity as a commercial piece of software.
SingularityCE is an Open Source project supported by Sylabs.
The three versions are similar today, but will diverge with time.
Apptainer is the most commonly adopted version of the scientific community.

Docker vs. Apptainer


Runs as a daemon process with superuser privileges	Runs as a regular user
Images stored centrally	Image files that you can move around.
Isolates the host and container file system by default	Containers have access to host file system
Well-supported on Mac, Linux and Windows	Limited support on Mac and Windows