Warning

Containers are not supported on Levante at this point.

How to containerIze your jupyter kernel?#

We have seen in this blog post how to encapsulate a jupyter notebook (server) in a singularity container . In this tutorial, I am going to describe how you can run a jupyter kernel in a container and make it available in the jupyter*.

Possible use case for this is to install a supported PyTorch version and work with jupyter notebooks.

Requirements#

  • singularity image supported on Levante

  • software packages

  • ipykernel

  • kernel.json

  • start-singularity-kernel.sh

How it works?#

The concept is simple, usually, software packages are located in conda/venv environements somwhere in Levante’s file system. The idea is to allow working with packages from a singularity container.

Pull the image#

There are lot of a predefined images with installed software packages. You don’t need to build it from scratch. These images must be supported on Levante.

In this tutorial, I am using this image with the following software packages:

  • jupyter

  • jupyterlab

  • matplotlib

  • pandas

  • ipykernel

  • xarray

  • intake-esm

  • intake-xarray

You need to download this image to your (default) HOME directory on MIstral.

# load singularity module
module load singularity
# pull the image
singularity pull library://lev-hpc/jupyter/jupyter-kernel:latest

Create start-singularity-kernel.sh#

touch start-singularity-kernel.sh

chmod +x start-singularity-kernel.sh

The content should look like this:

#!/bin/bash

source /etc/profile
module purge

module load singularity

singularity exec --cleanenv /full/path/to/jupyter-kernel_latest.sif python -m ipykernel_launcher -f "$1"

Create kernel.json#

You need a kernel configuration file:

module load python3

python -m ipykernel install --user --name singularity-kernel --display-name="singularity-kernel"

This file must be modified to enable the connection to the singularity container. My favorite method is via a bash script (let’s name it start-singularity-kernel.sh).

{
 "argv": [
  "/path/to/start-kernel-singularity.sh",
  "{connection_file}"
 ],
 "display_name": "singularity-kernel",
 "language": "python"
}

Without a bash script#

If you like to write all modifications in kernel.json:

{
 "argv": [
   "singularity",
   "exec",
   "--cleanenv",
   "/path/to/jupyter-kernel_latest.sif",
   "python",
   "-m",
   "ipykernel",
   "-f",
   "{connection_file}"
 ],
 "language": "python",
 "display_name": "singularity-kernel"
}
Webpack and Django   Requested MovieWriter (ffmpeg) not available