Working with containers

27-Nov-2024

What is Docker?

  • Standardized packaging for software, dependencies as well as the operating system
  • Docker lets you create and run applications securely isolated in containers

Docker in life science

Docker can also be used to define software environments and settings for benchmarking studies:


CAMI Challenge: an independent evaluation of several tools in the field of metagenomics.

[…] We defined standards for submitting the software itself, along with parameter settings and required databases and implemented them in Docker container templates … 1

Docker in life science

Docker can also be used to define software environments and settings for benchmarking studies:


2nd CAMI Challenge

For reproducibility, participants could submit a Docker container containing the complete workflow, a Bioconda script or a software repository with detailed installation instructions […] 1

Docker nomenclature

  • A Docker image is a stand-alone executable package of software
  • A Docker container is an instance of a Docker image
  • A Dockerfile is a recipe used to build a Docker image
  • Docker Hub is an online service for hosting Docker images

An example: using a different OS

Check OS on local machine

$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64

Pull Ubuntu (Linux) image

$ docker pull ubuntu:16.04
16.04: Pulling from library/ubuntu
22dc81ace0ea: Pull complete
...
Digest: sha256:e348fbbea0e0a0e73ab0370de151e7800684445c509d46195aef73e090a49bd6
Status: Downloaded image for ubuntu:16.04

An example: using a different OS

Check OS on local machine

$ uname -a
Darwin johnsmbp.local 19.6.0 Darwin Kernel Version 19.6.0: [...] x86_64

Run the container interactively and check OS version

$ docker run -it ubuntu:16.04
root@407b0fd13fe5:/## uname -a
Linux 407b0fd13fe5 4.9.60-linuxkit-aufs [...] x86_64 GNU/Linux

Mounting volumes

Local project/ directory:

$ ls
|- doc/
|- data/
|- code/
|- logs/
|- results/
|- Snakefile
|- config.yml
|- environment.yml
|- Dockerfile
|- README.md

Docker container file system:

$ docker run \
    -v $PWD/data:/home/data \
    ubuntu:16.04 \
    ls

|- bin/
|- dev/
|- etc/
|- home/
|  |- data/
|- lib/
|- sys/
|- usr/

What can I use Docker for?

  • As an advanced environment manager
docker run -it -v $PWD:/home my_image /bin/bash
  • To package your code with the environment it needs
docker run \
    -v $PWD/data:/home/data \
    -v $PWD/results:/home/results \
    my_image snakemake report.pdf
  • To package a whole project with environment, code and data (e.g. to accompany a manuscript).
docker run \
    -v $PWD/results:/home/results \
    my_image snakemake report.pdf

What is Apptainer?

  • Another software for working with containers, similar to Docker
  • A Singularity image is contained in a single file, facilitating portability
  • Does not require root access, making it suitable for work at HPCs

What is Apptainer?

  • Apptainer used to be ‘Singularity’.
  • In 2021 Singularity joined the Linux Foundation and became Apptainer.
  • The company Sylabs still maintains Singularity as a commercial piece of software.
  • SingularityCE is an Open Source project supported by Sylabs.
  • The three versions are similar today, but will diverge with time.
  • Apptainer is the most commonly adopted version of the scientific community.

Docker vs. Apptainer
Runs as a daemon process with superuser privileges Runs as a regular user
Images stored centrally Image files that you can move around.
Isolates the host and container file system by default Containers have access to host file system
Well-supported on Mac, Linux and Windows Limited support on Mac and Windows

Questions?