With TM351 about to start, I thought I’d revisit the docker container approach to delivering the services required by the course to see if I could get something akin to the course software running in the cloud.
Copies of the dockerfiles used to create the images can be found on Github, with prebuilt images available on dockerhub (https://hub.docker.com/r/psychemedia/ou-tm351-*).
A couple of handy ones out of the can are:
- the customised Jupyter notebooks and course python distribution [psychemedia/ou-tm351-pystack]
- the version of OpenRefine we’re using in the course [psychemedia/ou-tm351-openrefine]
That said, at the current time, the images are not intended for use as part of the course…
The following docker-compose.yml file will create a set of linked containers that resemble (ish!) the monolithic VM we distributed to students as a Virtualbox box.
dockerui: container_name: tm351-dockerui image: dockerui/dockerui ports: - "35183:9000" volumes: - /var/run/docker.sock:/var/run/docker.sock privileged: true devmongodata: container_name: tm351-devmongodata command: echo mongodb_created #Share same layers as the container we want to link to? image: mongo:3.0.7 volumes: - /data/db mongodb: container_name: tm351-mongodb image: mongo:3.0.7 ports: - "27017:27017" volumes_from: - devmongodata command: --smallfiles mongodb-seed: container_name: tm351-mongodb-seed image: psychemedia/ou-tm351-mongodb-simple-seed links: - mongodb devpostgresdata: container_name: tm351-devpostgresdata command: echo created image: busybox volumes: - /var/lib/postgresql/data postgres: container_name: tm351-postgres environment: - POSTGRES_PASSWORD=PGPass image: psychemedia/ou-tm351-postgres ports: - "5432:5432" openrefine: container_name: tm351-openrefine image: psychemedia/tm351-openrefine ports: - "35181:3333" privileged: true notebook: container_name: tm351-notebook #build: ./tm351_scipystacknserver image: psychemedia/ou-tm351-pystack ports: - "35180:8888" links: - postgres:postgres - mongodb:mongodb - openrefine:openrefine privileged: true
Place a copy of the docker-compose.yml YAML file somewhere, from Kitematic, open the command line, cd into the directory containing the YAML file, and enter the command docker-compose up -d – the images are on dockerhub and should be downloaded automatically.
Refer back to Kitematic and you should see running containers – the settings panel for the notebooks container shows the address you can find the notebook server at.
The notebooks and OpenRefine containers should also be linked to shared folders in the directory you ran the Docker Compose script from.
Running the Containers in the Cloud – Docker-Machine and Digital Ocean
As well as running the linked containers on my own machine, my real intention was to see how easy it would be to get them running in the cloud and using just the browser on my own computer to access them.
And it turns out to be really easy. The following example uses cloud host Digital Ocean.
To start with, you’ll need a Digital Ocean account with some credit in it and a Digital Ocean API token:
(You may be able to get some Digital Ocean credit for free as part of the Github Education Student Developer Pack.)
Then it’s just a case of a few command line instructions to get things running using Docker Machine:
docker-machine ls #kitematic usess: default #Create a droplet on Digital Ocean docker-machine create -d digitalocean --digitalocean-access-token YOUR_ACCESS_TOKEN --digitalocean-region lon1 --digitalocean-size 4gb ou-tm351-test #Check the IP address of the machine docker-machine ip ou-tm351-test #Display information about the machine docker-machine env ou-tm351-test #This returns necessary config details #For example: ##export DOCKER_TLS_VERIFY="1" ##export DOCKER_HOST="tcp://IP_ADDRESS:2376" ##export DOCKER_CERT_PATH="/Users/YOUR-USER/.docker/machine/machines/ou-tm351-test" ##export DOCKER_MACHINE_NAME="ou-tm351-test" # Run this command to configure your shell: # eval $(docker-machine env ou-tm351-test) #Set the environment variables as recommended export DOCKER_TLS_VERIFY="1" export DOCKER_HOST="tcp://IP_ADDRESS:2376" export DOCKER_CERT_PATH="/Users/YOUR-USER/.docker/machine/machines/ou-tm351-test" #Run command to set current docker-machine eval "$(docker-machine env ou-tm351-test)" #If the docker-compose.yml file is in . docker-compose up -d #This will launch the linked containers on Digital Ocean #The notebooks should now be viewable at: #http://IP_ADDRESS:35180 #OpenRefine should now be viewable at: #http://IP_ADDRESS:35181 #To stop the machine docker-machine stop ou-tm351-test #To remove the Digital Ocean droplet (so you stop paying for it... docker-machine rm ou-tm351-test #Reset the current docker machine to the Kitematic machine eval "$(docker-machine env default)"
So that’s a start. Issues still arise in terms of persisting state, such as the database contents, notebook files* and OpenRefine projects: if you leave the containers running on Digital Ocean to persist the state, the meter will keep running.
(* I should probably also build a container that demos how to bake a few example notebooks into a container running the notebook server and TM351 python distribution.)