Introduction to Container Registries

What is a Container Registry?

Container registries are services for storing and distributing container images. They enable reproducible, portable, and scalable workflows by allowing users to share pre-built environments and applications. Popular registries include Docker Hub, GitHub Container Registry (GHCR), and Quay.io.

Images are typically docker images that follow the open container image format as they are the most interoperable (i.e. useable with multiple container platforms, e.g. docker, singularity/apptainer, podman, charliecloud, shifter, etc). These images can be pulled and run on any compatible system, ensuring consistent environments across development, testing, and production.

Key benefits:

Centralized storage for container images
Versioning and access control
Integration with CI/CD pipelines

Bioinformatic specific registries

Biocontainers: These are automatically built from bioconda packages as stand-alone containers. Custom images and multi-tool containers (mulled containers) are also buildable, but require learning the system.
Seqera Container Registry: A much more stream-lined experience to build multi-tool images where images are quickly built and stored in a public registry.

Building a docker image from a conda environment.

You can create containers with custom Conda environments by specifying a environment.yml file.

environment.yml

name: myenv
channels:
  - conda-forge
dependencies:
  - python=3.10
  - numpy
  - pandas

Dockerfile

FROM mambaorg/micromamba:1.4.9

COPY environment.yml /tmp/environment.yml
RUN micromamba create -y -n myenv -f /tmp/environment.yml
ENV PATH=/opt/conda/envs/myenv/bin:$PATH

Tip

Extract the conda lock file from this environment and save it alongside your environment.yml.

docker run --rm -it <image> micromamba env export --explicit -n myenv > myenv-conda.lock

Uploading Images to GitHub Container Registry

GitHub Container Registry (GHCR) allows you to store Docker images alongside your code repositories.

Authenticate with GHCR. First get a Github token. Ensure that your token has permission to write and manage packages on Github.
```
echo $GITHUB_TOKEN | docker login ghcr.io -u <your-github-username> --password-stdin
```

Build Your Docker Image

docker build -t ghcr.io/<your-github-username>/<image-name>:<tag> .

Push the Image

docker push ghcr.io/<your-github-username>/<image-name>:<tag>

Tip

Use repository or organization names to organize your images.

Update package attributes. When you push an image, it will initially be private. Find the image under the packages tab in your Github profile and select it. Here you can attach the container image to a repository, and what permissions that repository has on the image (for example one could use Github Actions to automatically update the image and version, so write permission is needed).

Select “Package settings” in the bottom right, to change the package visibility from private to public (anyone can pull the image).

Warning

Docker stores your credentials after logging in. If your token expires or is deleted and you receive an error trying to pull a public image like:

docker pull ghcr.io/nbisweden/fastk_genescopefk_merquryfk:1.2
Error response from daemon: Head "https://ghcr.io/v2/nbisweden/fastk_genescopefk_merquryfk/manifests/1.2": denied: denied

then you need to logout again from the ghcr.io registry.

docker logout ghcr.io

Publishing to Seqera Container Public Registry

Seqera provides a public registry for building and sharing containers, especially for bioinformatics workflows. While the web interface is relatively powerful and easy to use, one can build custom packages and upload them through the wave cli tool.

Install the wave cli. Make sure the downloaded file is executable.

chmod 755 wave-1.6.1-macos-arm64
./wave-1.6.1-macos-arm64 --help

Build and upload your container image to the Seqera Container Public Registry using the Wave CLI. The --freeze flag uploads to the public registry, and --await waits for the build to complete. If successful, the CLI will output the image URI.
```
./wave-1.6.1-macos-arm64 --conda-file conda-env.yml --freeze --await
```
Example output:
```
community.wave.seqera.io/library/myenv:70473abb25330df7
```

Use the generated container image in your Nextflow process by specifying the image URI in the container directive:

process MY_PROCESS {
    container 'community.wave.seqera.io/library/myenv:70473abb25330df7'

    // ... rest of process definition ...
}

Summary

Container registries enable sharing and reuse of container images.
GHCR and Seqera Container Public Registry are popular options for storing and distributing images.
Custom Conda environments can be built into containers for reproducible workflows.
Use these registries to streamline your bioinformatics pipelines and ensure reproducibility.