Using Selenium to Support Teaching and the Production and Maintenance of Teaching Materials?

At the OU, we tell ourselves lots of myths, but don’t necessarily act them out. I believe more than a few of them, not least the one that we’re a content factory. I also believe we used to be really innovative in our production methods, but I think that’s largely fallen by the wayside in recent years.

The following is an example of a few hours play, though each step has probably taken me longer to write up in this post than the documented proof of concept code for each step took to produce.

It’s based on a couple of observations about Selenium that I hadn’t fully grokked until I played with it over the weekend, as described in the previous post (a recipe for automating bulk uploads of Jupyter notebooks to nbgallery), and then a riff or two off the back of them.

First up, I think we can use this to support teaching in a several ways.

One of the strategies we use in the OU for documenting how to use software applications is to use narrated screencasts, which is to say, screen-recordings of how to use an application with a narrated audio track explaining what’s going, and/or overlaid captions.

I wrote my nbgallery script as a way of automating bulk uploads, but its not hard to see how it can also be used to help in the automation of a screencast:

In that case, I did a test run to see where the browser was opened, then used Giphy to record a video of that part of the screen as I replayed the script.

The last time I recorded one of these was a couple of years ago and as I recall was a bit of a faff as I read from a script to dub the audio (I’m not a natural when it comes to the studio; I’m still not that comfortable, but still find it easier, recording an ad libbed take, although this is may become a bit fiddly when trying at the same time to control an application with a reasonable cadence).

What might have been easier would have been to script the sequence of button presses and mouse actions (though mousing actions would be lost?)

That said, it is possible to script in some highlighting too…

For example:

import time

def highlight(element, sleep=1.0):
    """Highlights (blinks) a Selenium Webdriver element"""
    driver = element._parent
    def apply_style(s):
        driver.execute_script("arguments[0].setAttribute('style', arguments[1]);",
                              element, s)
    original_style = element.get_attribute('style')
    apply_style("background: yellow; border: 2px solid red;")

gives something like this:

A couple of different workflows are possible here.

Firstly, we could bake timings in and record a completely automated screen-capture using time,wait() commands to hold each step as long as we need (or long enough so an editor can easily pause the video at a particular point for as many frames as are required).

Alternatively, we could use the notebook to allow us to step through the automation of particular actions.

What’s more, the notebook could include a script. Here’s an example in a step-through style:

One of the big issues with creating assets such as these is knowing the storyboard — what you expect to see at each step. This is particular true if a software application or webpage is updated, and an automation script breaks.

At a technical level, knowing what the original paged looked like as HTML can help, but the best crib is often a view of the original rendered display.

Which makes me think: it’s trivial to grab a screenshot of each step and insert those back into the notebook?

Here’s a code fragment for that:

import tempfile
from IPython.display import Image

#Create a temporary file for now
imgfile = tempfile.mktemp(suffix='.png')

#Get a browser element - this would be any old step

#Grab a screenshot fo the browser

#Display the screenshot in the notebook

Not only can this help us document script at a step level, but it also sets up an opportunity to create a text document (rather than a video screencast) that describes what steps to do when.

Can we also record a video of the automation? Selenium appears not to offer that out of the can, but maybe ffmpeg can help (ffmpeg docs)? Alternatively this Selenium docker image looks to support video capture, though I don’t see offhand to drive it from Python?

I also wonder: do the folk who do testing use this sort of automation, and if so, why don’t they share the knowledge and scripts back with us as a way of helping automate production as well as test? After all, that’s where factories are useful: mechanisation / automation helps with the scaling.

Once we start thinking about creating sorts of media asset, it’s natural to ask: could we also create a soundtrack?

I don’t see why not…

For example, pyttx3 is a cross-platform text-to-speech application, albeit with not necessarily the best voice:

#!pip3 install pyobjc pyttsx3

import pyttsx3
engine = pyttsx3.init()

def sayItToMe(txt):
    ''' Simple text to speech. '''

We can explicitly create text strings, but I don’t see why we should also find a way of grabbing relevant text from markdown cells?

TXT = '''
The first thing we need to do is log in.

TXT = '''
Select the person icon at the top right of the screen.

element = driver.find_element_by_id("gearDropdown")

Okay, so that’s one way in which we may be able to make use of Selenium, as a way of creating reproducible scripts for creating documentation in a variety of media of how to use a particular web application or website.

How about the second?

I think that one of the claims made for using Scratch in our introductory computing module is that you can get it to control animated things, which can help novices see the actions of particular steps in an animated way originally designed to appeal to primary school children. (And yes, I am prepared to argue an androgogy vs. pedagogy thing, as well as a curriculum thing, about why I think we should have used BlockPy.)

If you want shiny, and animated, and perhaps a little a bit frightening, perhaps (surprisingly) useful, and contextualised by all sorts of other basic computing stuff, like how browsers work, and what HTML and the DOM are (so you can probably make synoptic claims too…), then automatically launching a browser from a script and getting it to click things and take pictures might be seen as a really cool, or fun, thing to do — did you know you can…? etc. — and along the way provide a foil for learning a bit about scripting too.


PS longtime readers will note the themes of this post fit in with a couple of oft-repeated ideas contained elsewhere in this blog. For example, the notion I’m trying to work up of “reproducible educational materials” (which also doubles as the automation of rich media assets, and which is something I think is useful from production, testing and maintenance perspectives in a content factory (though no-one else seems to agree:-(,l and the use of notebooks for everything (which again, most people I know think is just me going off on one again…:-(.

Bulk Jupyter Notebook Uploads to nbgallery Using Selenium

I’ve recently started looking at nbgallery [repo], “an enterprise Jupyter Notebook sharing and collaboration platform” written in Ruby. The gallery provides a range of tools, including:

  • a Solr powered notebook search engine;
  • a notebook “health check” (I haven’t tried this yet);
  • integration with Jupyter notebooks, so you can run notebooks (I haven’t tried this yet).

One thing that seems to be lacking is the ability to bulk upload files (for example, contained in a zip file). I haven’t spotted an API either, or a Python wrapper to provide a de facto API. This makes a proper test over lots of notebooks tricky…

The notebook upload is a two step process.

The first step requires selection of a notebook, and a required acknowledgement of rights:

The second provides and opportunity to submit a required title and non-null description and a (repeated) rights acknowledgement:

The upload process utilises a multi-part form.

To upload a notebook, a user needs to be logged in.

Creating a new user requires an email confirmation step, which means you need to set up email server details in the docker-compose.yml file. I used my OU ones:


My usual approach for automating this sort of thing would be to have a go with mechanical soup or mechanize, but on a quick first attempt using both of those, I couldn’t get the scraper to work.

Instead, I took the opportunity to have a play with Selenium With Python, a Python wrapper for the Selenium web testing framework. This provides a set of Python functions for automating the launching of a web-browser (Chrome, Safari, Firefox, etc) and the automated clicking of pages viewed within that automated browser.

The full script I used can be found here.

The initialisation looks like this:

from selenium import webdriver

#Selenium package includes several utilitities
# for waiting until things are ready
from import By
from import WebDriverWait
from import expected_conditions as EC

driver = webdriver.Chrome()

#Allow the driver to poll the DOM for up to 10s when
# trying to find an element

#We might also want to explicitly define wait conditions
# on a particular element
wait = WebDriverWait(driver, 10)


The login function looks something like this:

def nbgallery_login(driver, wait, user, pwd):
    ''' Login to nbgallery.
        Return once the login dialogue has disappeared.


    element = driver.find_element_by_id("user_email")


    element = driver.find_element_by_id("user_password")


The first form script looks like this:

    #path is full path to file
    if not path.endswith('.ipynb'):
        print('Not a notebook (.ipynb) file? [{}]'.format(path))

    #Part 1

    element = wait.until(EC.element_to_be_clickable((By.ID, 'uploadModalButton')))


And the script to handle the second part of the form looks like this:

    #Part 2
    element = driver.find_element_by_id("stageTitle")

    #Is there notebook metadata we can search for title?
    if not title:
        title = path.split('/')[-1].replace('.ipynb','')

    element = driver.find_element_by_id("stageDescription")

    #Is there notebook metadata we can search for description?
    #Any other notebook metadata we could make use of here?
    #Description needs to be not null
    desc= 'No description.' if not desc else desc

    element = driver.find_element_by_id("stageTags-tokenfield")

    #Handle various tagging styles
    #Is there notebook metadata we can search for tags?
    tags = '' if not tags else tags
    if isinstance(tags, list):
    tags = tags if tags.endswith(',') else tags+','

    element.send_keys(tags) #need the final comma to set it?

    if private:


    #Wait for new page to load

Here’s how it plays out:

There’s still stuff that could be added — error trapping for duplicate notebooks, for example — but I think this is enough to let me upload a complete set of course notebooks and see how useful nbgallery is as a way of presenting notebooks.

If it is, and I get the Jupyter notebook server integration working, then I wonder: would it be useable as a notebook navigator in the TM351 VM? It’d probably need really tight integration with the notebook server so that when notebooks are saved they are also committed to the gallery?

Custom Charts – RallyDataJunkie Stage Table, Part 1

Over the last few evenings, I’ve been tinkering a bit more with my stage table report for displaying stage split times, using the Dakar Rally 2019 timing data as a motivator for it; this is a useful data set to try the table out with not because the Dakar stages are long, with multiple waypoints (that is, splits) along each stage.

There are still a few columns I want to add to the table, but for now, here’s a summary of how to start reading the table.

Here’s stage 3, rebased on Sebastien Loeb; the table is ordered according to stage rank:

The first part of the chart has the Road Position (that is, stage start order) using a scaled palette so that out of start order drivers in the ranking are highlighted. The name of the Crew and vehicle Brand follow, and a small inline step chart that shows the evolution of the Waypoint Rank of each crew (that is, their rank in terms of stage time to that point, at each waypoint). The upper grey bar shows podium ranks 1 to 3, the lower grey line is tenth. If a waypoint returns an NA time, we get a break in the line.

Much of the rest of the chart relies on “rebased” times. So what do I mean by “rebased”?

One of the things the original data gives us the stage time it took each driver to get to each way point.

For example, it took Loeb 18 minutes dead to get to waypoint 1, and Peterhansel 17m 58. Rebasing this relative to Loeb suggests Loeb lost 2s to Perterhansel on that split. On the other hand, Coronel took 22:50, so Loeb gained 290s.

Rebasing times relative to a particular driver finds the time difference (delta) between that driver and all the other drivers at that timing point. The rebased times show for each driver other than the target driver are thus the deltas between their times and the time recorded for the target driver. The rebased time display was developed to be primarily useful to the driver with reference to who the rebased times are calculated.

So what’s going on in the other columns? Let’s rebase relative to Loeb.

Here’s what it looks like, again;

The left hand  middle of the table/chart shows time taking in making progress between waypoints.

To start with we have the Stage Gap of each driver relative to Loeb. This is intended to be read from the target driver’s perspective, so where a driver made time over the target driver, we colour it red to show our target lost time relative to that driver. If a driver was slower than the target driver (the target made up time), we colour it green.

The Stage Gap is incremental, based on differences between drivers of based on the total time in stage at each waypoint. In the above case, Loeb was losing out slightly to the first two drivers at the first couple of waypoint, but was ahead of the third place driver. Then something went bad and a larget amount of time was lost.

But how much time? That what the inline bar chart cells show: the time gained / dropped going from one waypoint to the next. The D0_ times capture differences in the time taken going from one split/waypoint to the next. The horizontal bar chart x-axis limits are set on a per column basis, so you need to look at the numbers get a size of how much time gained/lost they represent. The numbers are time deltas in seconds. I ummed and ahhed about the sign of these. At the moment, a positive time means the target (Loeb) was that much time slower (extra, plus) than the driver indicated by the row.

Finally, the Pos column is rank position at the end of the stage.

If we look down the table, around Loeb, we see how Loeb’s times compare to the drivers who finished just ahead —and behind— hi. For drivers ahead in the ranking, their Stage Gap will end up red at the end of the stage, for drivers behind, it’ll be green (look closely!)

Scanning the D0_ bars within a column, it’s obvious which bits of the stage Loeb made, and dropped, time.

The right hand side of the figure considers the stage evolution as a whole.

The Gap to Leader column shows how much time each driver was behind the stage leader at each waypoint (that is, at each waypoint, rank the drivers to see who was quickest getting to that point).

Along with the Waypoint Rank, the Road Position and Gap to Leader, this is the only aspect of the table that is relative to the driver associated with that row: it helps our target (Loeb) put each other driver’s performance on the stage in the context of the overall stage rankings. The dot marker indicates the gap to leader at the end of the stage.

The 0N_ columns show the time delta on stage between each driver and Loeb, which is the say, the delta between the accumulated stage time for each driver at each waypoint. The final column records the amount of time, in seconds, gained or lost by Loeb relative to each driver in the final stage ranking (penalties excepted).

Looking at the table aound Loeb we see the column entries are empty except for the Gap to Leader evolution.

The original version of this chart, which I was working up around WRC 2018, also includes a couple more columns relating to overall rally position at the start and end of the stage. Adding those is part of my weekend playtime homework!

Converting Pandas Generated HTML Data Tables to PNG Images

Over the weekend, I noticed the Dakar 2019 rally was on, which resulted in my spending Sunday evening putting a scraper together to grab timing data down from the official website (notebook code here).

The data on its own is all a bit “so  what?”; it only comes alive when you start playing with it. One of the displays I was still tinkering with at the end of last year’s WRC season was a tabular stage report that tries to capture a chunk of information from stage timing, such as split times, so it made sense to start riffing on that.

The Rally Dakar timing screen presents split times like this:

You can get a view with either of the running time in stage at each split / waypoint, or the gap, which is to say, the time difference to the person who was fastest at each split. (I think sometimes Gap times may report the time difference to the person who ranked first on the stage overall, rather than the gap to the person ranked first at a particular split.)

One of the things that interests me (from a data storytelling point of view) is how much time a driver gains, or loses, within a split compared to other drivers. We can use this to spot parts of the stage where a driver has hit a problem, or pushed hard.

The sort of display I’ve been working up looks, at least with the Dakar data, like this so far (there are a few columns missing compared to my WRC tables, but there’s also an extra one: the in-line bimodal sparkline chart).

This particular view displays split times rebased relative to Peterhansel (it’s easy enough to generate views rebased relative to any other specified driver). That is, the table shows how much time Peterhansel gained/lost relative to each other driver at each waypoint. The table is ordered by stage rank. The columns on the left show how much time was gained/lost going from one waypoint to the next. The columns on the right show how the gap relative to each driver evolved over the stage. The inline chart tracks the gap evolution.

The table is a styled pandas table, rendered as HTML. After applying styling, you can get a preview in a notebook using something of the form:

from IPython.display import display, HTML
display( HTML( ) )

I’ve previously posted a recipe for Grabbing Screenshots of folium Produced Choropleth Leaflet Maps from Python Code Using Selenium so here’s the latest iteration of my code fragment (which built on the previous example) for taking a chunk of HTML and using selenium to open it in a browser and grab a screenshot of it.

The code is h/t to several Stack Overflow posts.

import os
import time
from selenium import webdriver

def setup_screenshot(driver,path):
    ''' Grab screenshot of browser rendered HTML.
        Ensure the browser is sized to display all the HTML content. '''
    # Ref:
    original_size = driver.get_window_size()
    required_width = driver.execute_script('return document.body.parentNode.scrollWidth')
    required_height = driver.execute_script('return document.body.parentNode.scrollHeight')
    driver.set_window_size(required_width, required_height)
    # driver.save_screenshot(path)  # has scrollbar
    driver.find_element_by_tag_name('body').screenshot(path)  # avoids scrollbar
    driver.set_window_size(original_size['width'], original_size['height'])

def getTableImage(url, fn='dummy_table', basepath='.', path='.', delay=5, height=420, width=800):
    ''' Render HTML file in browser and grab a screenshot. '''
    browser = webdriver.Chrome()

    #Give the html some time to load
    imgfn = '{}/{}'.format(basepath, imgpath)
    imgfile = '{}/{}'.format(os.getcwd(),imgfn)

    return imgpath

def getTablePNG(tablehtml, basepath='.', path='testpng', fnstub='testhtml'):
    ''' Save HTML table as: {basepath}/{path}/{fnstub}.png '''
    if not os.path.exists(path):
        os.makedirs('{}/{}'.format(basepath, path))
    fn='{cwd}/{basepath}/{path}/{fn}.html'.format(cwd=os.getcwd(), basepath=basepath, path=path,fn=fnstub)
    with open(fn, 'w') as out:
    return getTableImage(tmpurl, fnstub, basepath, path)

#call as: getTablePNG(s)
#where s is a string containing html, eg s =

The png image is saved as an image file that can be embedded in other HTML pages, shared via soshul meeja, etc…

Docker Housekeeping _ Removing Old Images and Containers

Some handy commands for tidying up old Docker images and containers…

Remove Named Images

docker rmi `docker images --filter 'dangling=true' -q --no-trunc`

Remove Images With a Particular Name Pattern

Ish via here.

For example, removing repo2docker default named containers which start r2d:

docker images | awk '{ print $1,$2, $3 }' | grep r2d | awk '{print $3 }' | xargs -I {} docker rmi {}

For added aggression, use rmi -f {}.

Remove Containers With a Particular Name Pattern

Via here.

docker ps -a | awk '{ print $1,$2 }' | grep r2d | awk '{print $1 }' | xargs -I {} docker rm {}

Remove Exited Containers

docker ps -a | grep Exit | cut -d ' ' -f 1 | xargs docker rm

PS lots of handy tips here:

Authenticated OpenRefine Server on Digital Ocean, Redux

Following on from yesterday’s recipe showing how to Running OpenRefine On Digital Ocean Under Simple Auth, here’s an even easier way that doesn’t require ssh and doesn’t require console access: just copy and paste some set-up info into a form on the Digital Ocean droplet creation page.

Every little everything helps… this link will set new users up with $100 of free credit on Digital Ocean; somewhere down the line I may get a small amount of affiliate link powered Digital Ocean credit to help keep my own server costs covered…

The Digital Ocean droplet creator has an option to add a User data start-up script when the droplet is created (docs).

This means we can provide a script that will download and install everything we need, including a file to define a service that will run OpenRefine.

Copy and paste the script in the gist that can be found here into the user data area before you create the droplet. The code will be executed once the droplet is up and running and should install the nginx proxy and the OpenRefine application. A default user (test) and password (letmein) are defined in script.

If you are trusting of the gist, you can install it more succinctly from the gist repository. You can also define your own user and password. To take this installation route, use the code that follows below in the userdata area:

Here’s the code…:


#We can over-ride the default credentials

#Get the latest version of installer script and run it

source <( curl -s )

(There should be no spaces between the less than and open bracket characters in the source command.)

For an explicit link to a particlar version of the script, go to the gist, and copy the link for the raw version of the latest file or the version you want.

Note that this route requires you to place considerable trust in the publisher of the remote installation script. Do you really trust a file with such a dodgy filename?

It will probably take two or three minutes to download and install everything, so don’t be too worried if you don’t see anything at the provided IP address immediately. Just keep refreshing the page…

The first sign of life you should see is the nginx default page…

Wait a few moments and reload the page… Next up is a holding page as the OpenRefine application is installed and the service started…

This page should automatically refresh itself every 5 seconds or so. When the service is up and running, you should get a page like this:

Click the link and you should be prompted with the the user/password authenticator challenge. Provide the username/password and you should be taken to your OpenRefine server.

AutoStarting A Headless OpenRefine Server in MyBinder Using Repo2Docker and a start Config File

When I first started using MyBinder in 2015, it came with the option of autostarting selectable services — PostgreSQL and a Spark server — within the container along with the Jupyter notebook server (early review). I’m not sure when it disappeared (the Github repo commit history should show it, if anyone’s feeling forensic investigative and wants to let me know via a comment) but for some time I’ve been wondering how to start my own services, such as a database server, or OpenRefine server, in a Binderised container.

I guess I shoulda read the docs again…

…although the pace of change with: a) documented features; b) undocumented features, means it can be hard to keep up with what’s possible in the Jupyterverse (I’m nowhere close to cracking that with the TrackingJupyter in its current formulation…).

So via this issue, some handy leads…

repo2docker start

Binderised containers are built using repo2docker. A new-to-me config file for repo2docker is the start file, “a script that can contain simple commands to be run at runtime. If you want this to be a shell script, make sure the first line is #!/bin/bash. The last line must be exec "$@" equivalent”.

So for example, if we want to autorun a headless OpenRefine instance in MyBinder that I can access via a notebook using an OpenRefine Python client, (see an example notebook here), we can just add the following start file to the repo:


#Start OpenRefine
nohup openrefine-2.8/refine -p 3333 -d OPENREFINE_DIR > /dev/null 2>&1 &

#Do the normal Binder start thing here...
exec "$@" 

Demo here: Binder

PS thanks as ever to the ever helpful Jupyter devs for putting up with my “issues”…

PPS I got stuck trying to generalise the running auto-starting, arbitrary services thing any further. You can see the state of my ignorance here. As is often the case, “permissions” makes me realise I don’t really understand how any of this stuff actually works. That’s why I tend to work: a) locally, b) as root…!