[Not] Running Jupyter Notebook Containers Via Jupyterhub Container Under Kubernetes Under Docker on My Desktop

…(sic).

One of the things I’ve been exploring lately (actually, that my colleague Rod Norfor in IT has been exploring lately) has been a way of offering a self-service, disposable Jupyter notebook service, pre-packaged with customised, pre-prepared teaching notebooks for students to work through. The hope is that this will go live to some TM112 students later this month.

The solution we’ve come up with is running Jupyterhub under Kubernetes on Microsoft Azure Cloud. The Jupyterhub server serves a prebuilt, public Docker container pulled from Docker Hub. (As a back-up, the same notebooks can be run via Microsoft Azure Notebooks cloned from a library on Microsoft Azure Notebooks or from the Github repo used to build the notebook container served by Jupyterhub.) On the to do list is to explore whether we should use a private registry or pull from an OU hosted registry, either private or public.

One thing we have noticed is that tagging the source notebook container image as latest breaks things with Kubernetes if we try to serve an updated image. The solution is to tag each updated version of the container differently (?and restart the Jupyterhub server?). The image is rebuilt using the same tag as a Github release tag, and the build triggered from a Github release. The Github release is handled manually.

As things currently stand, authentication is required to get into the OU VLE in the normal way, and then a link to the Jupterhub server containing a secret is used to take students to the Jupyterhub server. From there, users self-start the Jupyter notebook container.

In the default setup, dummy auth is in place, letting users through an explicit check with any user/password combination:

 

Thanks to Rod, the Azure route appears to be working so far, but  I thought I’d also try to set up a local development environment on my desktop.

The Docker Community Edition Edge release](https://www.docker.com/kubernetes) allows you to run Kubernetes locally on your desktop under Docker. Having got Kubernetes installed and up running straightforwardly enough, it was time to give Jupyterhub a go in its default state. The following recipes is via a get started crib from David Currie and the Zero to JupyterHub with Kubernetes docs:

## Install helm
## - download and install appropriate distribution (I grabbed the Mac distribution from [kubernetes/helm](https://github.com/kubernetes/helm), unzipped it, and put the app in my `Applications` folder, which is in my path)

##Check we can see it
helm

## Initialise helm
helm init

## Create a working directory, just in case!
mkdir code/dockerk8sjupyterhub
cd code/dockerk8sjupyterhub

## Set up things to work with my local docker powered kubernetes service
kubectl config use-context docker-for-desktop
kubectl get nodes

##Tiller is already installed - I'm not sure if this is necessary?
helm init --service-account tiller

## Security voodoo
kubectl --namespace=kube-system patch deployment tiller-deploy --type=json --patch='[{"op": "add", "path": "/spec/template/spec/containers/0/command", "value": ["/tiller", "--listen=localhost:44134"]}]'

## Create a config file
touch config.yaml
## Create a secret hash value
openssl rand -hex 32
## And add it to the YAML file:
nano config.yaml
|------
proxy:
  secretToken: "MY_TOKEN"
|------

## Add helm chart
helm repo add jupyterhub https://jupyterhub.github.io/helm-chart/
helm repo update
helm install jupyterhub/jupyterhub --version=v0.6 --name=thhelm1juphub --namespace=ouseful -f config.yaml
##locally:
##helm install path/to/my/files
##helm install my_chart.tgz

## When it's running, find the IP address
kubectl --namespace=ouseful get svc proxy-public
|------
NAME           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
proxy-public   10.104.156.186   localhost     80:32292/TCP,443:32125/TCP   1h
|------
## In this case, we can find Jupyterhub at: 127.0.0.1:80

##Bring everything down with:
helm delete ouseful --purge
kubectl delete namespace ouseful

One of the things I’d like to do is explore things like customise the notebook container start page:

…and the holding page as the server starts up:

and the error pages (at the moment, this is as far as I can get:-(

(I seem to remember there were issues setting up the helm chart to work with Azure the first time, so I’m hoping there’s a similar simple fix to get my local environment working. If you can point me to possible solutions via the comments, that would be appreciated:-)

I can also easily raise a 404 error page:

It looks as if the template pages are in the jupyterhub repo here: share/jupyterhub/templates so while things are still broken, now might be a good time for me to try some simple (branded) customisations of them.

At the moment, there are two things I need to clarify for myself (help via the comments appreciated!):

• where is the path to the Jupyterhub container specified? Ideally, I want to build a local image containing customised start/error pages.
•  where is the path to the Jupyter notebook image served by Jupyterhub specified so that I can serve my own test containers?

Clarification here: only one container is required – the one that will be served – but :

The Docker image you are using must have the jupyterhub package installed in order to work. Moreover, the version of jupyterhub must match the version installed by the helm chart that you’re using. For example, v0.5 of the helm chart usesjupyterhub==0.8.

To use a new image, tweak the config.yaml file, for example:

singleuser:
  image:
    name: jupyter/scipy-notebook
    tag: c7fb6660d096