Using Selenium to Support Teaching and the Production and Maintenance of Teaching Materials?

At the OU, we tell ourselves lots of myths, but don’t necessarily act them out. I believe more than a few of them, not least the one that we’re a content factory. I also believe we used to be really innovative in our production methods, but I think that’s largely fallen by the wayside in recent years.

The following is an example of a few hours play, though each step has probably taken me longer to write up in this post than the documented proof of concept code for each step took to produce.

It’s based on a couple of observations about Selenium that I hadn’t fully grokked until I played with it over the weekend, as described in the previous post (a recipe for automating bulk uploads of Jupyter notebooks to nbgallery), and then a riff or two off the back of them.

First up, I think we can use this to support teaching in a several ways.

One of the strategies we use in the OU for documenting how to use software applications is to use narrated screencasts, which is to say, screen-recordings of how to use an application with a narrated audio track explaining what’s going, and/or overlaid captions.

I wrote my nbgallery script as a way of automating bulk uploads, but its not hard to see how it can also be used to help in the automation of a screencast:

In that case, I did a test run to see where the browser was opened, then used Giphy to record a video of that part of the screen as I replayed the script.

The last time I recorded one of these was a couple of years ago and as I recall was a bit of a faff as I read from a script to dub the audio (I’m not a natural when it comes to the studio; I’m still not that comfortable, but still find it easier, recording an ad libbed take, although this is may become a bit fiddly when trying at the same time to control an application with a reasonable cadence).

What might have been easier would have been to script the sequence of button presses and mouse actions (though mousing actions would be lost?)

That said, it is possible to script in some highlighting too…

For example:

import time

def highlight(element, sleep=1.0):
    """Highlights (blinks) a Selenium Webdriver element"""
    driver = element._parent
    def apply_style(s):
        driver.execute_script("arguments[0].setAttribute('style', arguments[1]);",
                              element, s)
    original_style = element.get_attribute('style')
    apply_style("background: yellow; border: 2px solid red;")

gives something like this:

A couple of different workflows are possible here.

Firstly, we could bake timings in and record a completely automated screen-capture using time,wait() commands to hold each step as long as we need (or long enough so an editor can easily pause the video at a particular point for as many frames as are required).

Alternatively, we could use the notebook to allow us to step through the automation of particular actions.

What’s more, the notebook could include a script. Here’s an example in a step-through style:

One of the big issues with creating assets such as these is knowing the storyboard — what you expect to see at each step. This is particular true if a software application or webpage is updated, and an automation script breaks.

At a technical level, knowing what the original paged looked like as HTML can help, but the best crib is often a view of the original rendered display.

Which makes me think: it’s trivial to grab a screenshot of each step and insert those back into the notebook?

Here’s a code fragment for that:

import tempfile
from IPython.display import Image

#Create a temporary file for now
imgfile = tempfile.mktemp(suffix='.png')

#Get a browser element - this would be any old step

#Grab a screenshot fo the browser

#Display the screenshot in the notebook

Not only can this help us document script at a step level, but it also sets up an opportunity to create a text document (rather than a video screencast) that describes what steps to do when.

Can we also record a video of the automation? Selenium appears not to offer that out of the can, but maybe ffmpeg can help (ffmpeg docs)? Alternatively this Selenium docker image looks to support video capture, though I don’t see offhand to drive it from Python?

I also wonder: do the folk who do testing use this sort of automation, and if so, why don’t they share the knowledge and scripts back with us as a way of helping automate production as well as test? After all, that’s where factories are useful: mechanisation / automation helps with the scaling.

Once we start thinking about creating sorts of media asset, it’s natural to ask: could we also create a soundtrack?

I don’t see why not…

For example, pyttx3 is a cross-platform text-to-speech application, albeit with not necessarily the best voice:

#!pip3 install pyobjc pyttsx3

import pyttsx3
engine = pyttsx3.init()

def sayItToMe(txt):
    ''' Simple text to speech. '''

We can explicitly create text strings, but I don’t see why we should also find a way of grabbing relevant text from markdown cells?

TXT = '''
The first thing we need to do is log in.

TXT = '''
Select the person icon at the top right of the screen.

element = driver.find_element_by_id("gearDropdown")

Okay, so that’s one way in which we may be able to make use of Selenium, as a way of creating reproducible scripts for creating documentation in a variety of media of how to use a particular web application or website.

How about the second?

I think that one of the claims made for using Scratch in our introductory computing module is that you can get it to control animated things, which can help novices see the actions of particular steps in an animated way originally designed to appeal to primary school children. (And yes, I am prepared to argue an androgogy vs. pedagogy thing, as well as a curriculum thing, about why I think we should have used BlockPy.)

If you want shiny, and animated, and perhaps a little a bit frightening, perhaps (surprisingly) useful, and contextualised by all sorts of other basic computing stuff, like how browsers work, and what HTML and the DOM are (so you can probably make synoptic claims too…), then automatically launching a browser from a script and getting it to click things and take pictures might be seen as a really cool, or fun, thing to do — did you know you can…? etc. — and along the way provide a foil for learning a bit about scripting too.


PS longtime readers will note the themes of this post fit in with a couple of oft-repeated ideas contained elsewhere in this blog. For example, the notion I’m trying to work up of “reproducible educational materials” (which also doubles as the automation of rich media assets, and which is something I think is useful from production, testing and maintenance perspectives in a content factory (though no-one else seems to agree:-(,l and the use of notebooks for everything (which again, most people I know think is just me going off on one again…:-(.

Editing Text in the Browser

Via the Guardian Developer blog, a post — Leaving Scribe — describing how the Guardian is moving away from its Scribe in-browser text editor to a new one based on ProseMirror, an open-source toolkit “for building rich-text editors on the web” that is also used by the New York Times.

In-browser editors are not something I know much (i.e. anything) about, but the Leaving Scribe post provides a handy review of what’s good to know (like how markup is handled). Go and read it now…

It seems like the Guardian folk have many of the same issues as we do in the OU. For example:

Another area where HTML as a model falls down is editor-only annotations (markup that helps the writer but is detrimental to the reader). Take for example the need to highlight a word in the text that meets some criteria (a suggested tag, or some legal issue around using this word). You may want to show an inline annotation to ask the editor whether they want to add this as a tag.

The problem here is that now we have data that is not part of the document, and yet it is modelled as part of our document. This is technically solvable but again, the DOM API is not well suited for handling this sort of data modelling, especially when the usage of these features becomes more complex. As you start to force more complex features through an HTML data model you have to do more and more work to get around HTML’s limitations around modelling a rich text document and you hit more and more of the browser inconsistencies.

Features of ProseMirror based editors apparently include collaborative editing and an extensible schema. This last one is interesting from an OU perspective, because we have a workflow in which content is published from an internal XML document feedstock.

The important difference between Scribe and ProseMirror is that ProseMirror implements its own model layer that has a one-to-one mapping from semantics to the model, and an API that is made with document transformation in mind – not least collaborative editing.

An image representation of ProseMirror's model

In ProseMirror, inline content is flat rather than a tree, which means operations like changing styles on text don’t require any tree manipulation. And while nodes (h1, p, blockquote etc.) are still modelled as a tree but again, this accurately models how users think about things like paragraphs and lists, and it’s almost always how they’re rendered when consuming an article.

I’m not sure if the halted OU Create project was using ProseMirror? (I never really found out any technical details and I was banned from posting screenshots or discussing [di(scu)ssing?!] it in public!;-)

We hope in time to be able to get our editor to a point that it is able to be open-sourced but we’ll only do this if we believe we have the documentation and resource in place for that to be useful to users outside the Guardian.

Ah ha… It’d be nice if an OU solution could work in an open-sourcey way, or perhaps join forces with others to get such code out there…

One of the things I’ve been pondering lately is how to generate OU XML from Jupyter notebooks, as well how to demonstrate rich text authoring in notebooks using things like the jupyter-wysiwyg editor (I wonder how easy it is to modify that extension to work with other rich editors?)

So I wonder a couple of things:

  • how easy would it be to extend ProseMirror to support the OU XML schema?
  • could this customised editor then be used as a rich editor inside a Jupyter notebook markdown cell? (Would it need tweaks to the markdown2html renderer, or an OU-XML2HTML previewer?)

I’m also thinking that OU-XML has a lot of metadata elements which could be embedded as notebook metadata, with just a subset of the OU-XML being supported within the markdown cells. (Markdown cells could also have metadata associated with them.)

I think we could probably get a clunky workflow going quite quickly for authoring OU-XML docs from within Jupyter notebooks if anyone else was interested in exploring it with me…

First Play With nbgallery

Having hacked together a bulk uploader for nbgallery and uploaded the TM351 notebooks to a test environment, I’m now in a position to start having a play with it.

All public notebooks are searchable, so how does the search fare?

The search box top right gets a little bit lost in the search results listing. It could be handy to at least print out the search string (“Searching for: …”) at the top of the results list, if not making the search box larger and in a more central location. The search results themselves take the form of the name / description/tag of each hit (i.e. the notebook metadata) along with a fragment showing how the search terms appeared in context within the notebook.

Some of my earlier experiments on notebook search here and here also show context.

A range of options are provided for ordering the results. Trending looks like it could be interesting (this is based on recent views, presumably), for example where students are searching notebooks relevant to the current week’s study.

That said, we can also display notebooks by tag, so it’s easy enough to display notebooks associated with a particular week’s study if we tag notebooks by study week:

(One thing I noticed zooming out on the page to grab the above screenshot is that the font size of the notebook titles doesn’t seem to respond to the zoom level; it would probably be worth checking to see if there are other accessibility issues.)

If we click through on a result, we see a list of related notebooks followed by a preview of the notebook. (nbgallery strips out all cell outputs on upload, so no cell outputs are displayed).

To search through the preview, we can use a normal browser in-page search (ctrl/cmd-F).

A range of options are provided to support community activity around a notebook for logged in users, including the ability to “star” a notebook, provide feedback or add a comment:

Logged in users can also click on the notebook tags to edit them.

Via the Further options menu, users can view various notebook metrics, email a notebook, or propose a change request:

The metrics available include number of views, runs, stars and the edit history.

If comments have been provided, the number indicator by the comment flag shows how many comments have been received, although this only appears on the notebook page. There doesn’t appear to be an indicator of how many comments are associated with a notebook on the search results page, nor did I spot a general “recent comments” feed anywhere.

When you post a comment, there is no indication that you have done so and the form remains in place. You need to close it manually. (Hitting “Post Comment” again just pops up a “can’t do that” alert on the grounds that you’re trying to post a duplicate comment.)

The comments themselves look as if they are an ordered (rather than threaded) list. It also looks like any signed in used can edit anybody else’s comment?

Users who aren’t signed in can download a notebook, but not star it, comment on it, modify the tags etc.

When I tried to add feedback, I got an error:

I’m not sure if there are settings I need to tweak to address that?

Logged in users can also run a notebook from nbgallery via an associated notebook server. (I’d prefer it if the Run in Jupyter flash wasn’t displayed if there isn’t a linked notebook server available for the logged in user.) For example, running a notebook server on  port 443 on the same host as nbgallery using the nbgallery notebook container:

docker run --rm -p 443:443 -e "NBGALLERY_URL=http://localhost:3000" -e "NBGALLERY_CONFIG_TOKEN=letmein" nbgallery/jupyter-alpine

starts a notebook server with the nbgallery extension pre-installed.

We can view the notebook server homepage on https://localhost:443 and log into it using the token-as-password letmein. Running the container in the way described above also gives permission for the nbgallery server running in on http://localhost:3000 to open notebooks via the notebook server.

Within nbgallery itself, a logged in user can associate one of more Jupyter environments via the user menu:

Each environment is given a name and the URL of the associated notebook server (in this case, https://localhost:443):

When a notebook server is associated with a user, notebooks can be opened from nbgallery within the notebook server.

If we create a new notebook in the linked notebook server, we can upload it to nbgallery, adding a title, description and optional tags as in a manual notebook upload step:

If we modify the notebook that is linked to one in the gallery (that is, that has been uploaded to the gallery or launched from the gallery), we can save a change to the gallery or submit a change request:

When uploading a new version, you can add tags but not additional comments such as a commit message:

Viewing the notebook details in nbgallery, we can see a summary of the change history:

We can also click through to a preview of each version of the notebook:

(The revision number doesn’t appear in the change history though, so it can be hard to reconcile a particular version with it’s appearance in the change history listing.)

A logged in user can make a change request to someone else’s notebooks by uploading a new version of them or by opening the notebook in the linked notebook server and submitting a change request:

When I submitted the change request, I got an error form in response, but it looks like the change request was made, as this listing of Change Requests from the user menu suggests:

An exclamation mark by the user menu also identifies that change requests are pending.

Viewing the change request provides a view over the current version of the notebook and the proposed changes. Notebooks can be viewed alongside each other or the diffs can be viewed:

The thumbs up/down indicators are used to accept or deny a change request, along with a brief comment:

Accepted changed notebooks are used to replace the current version of the notebook, and the change logged in the change history. Denied change requests are recorded as such in the change requests list, with a link to the version of the notebook containing the unsuccessfully proposed changes:

If feedback was provided, a comment icon identifies its presence and pops up the feedback in a tooltip when hovered over.

Health stats for linked and run notebooks are supposed to be available, but I couldn’t get those to work (as far as the health stat reports were concerned, the notebooks were never run no matter how many times I ran them), so maybe I’m missing something there in the setup too? [UPDATE: health settings run with a flag set: notebook instrumentation docs; specifically, -e NBGALLERY_ENABLE_INSTRUMENTATION=1 in the docker command line.]

I’m not sure how well this would work for managing TM351 notebooks compared to out current Github workflow (which I should write up somewhere). The error responses (whether they’re valid or not) for change requests and feedback are confusing, and I’m not sure how the feedback is handled if and when it works. Not being able to easily spot new comments easily (unless I’m missing something) could be a bit of a pain. That said, the proof would be in the testing-through-use, so I’ll maybe give it a week or two’s trial with some of my own notebook workflows.

In terms of use with students, it could be useful to provide a version of nbgallery with notebooks runnable by students without them having to log in to it. It could also be useful if notebooks could be run ‘inline’ from the notebook preview pages, for example using something like ThebeLab or Voila, particularly if a particular Binderhub repo / config could be specified in metadata somewhere.

Quick Way in to Hacking Legacy OU Course Materials Using Markdown

By some arcane process, OU course materials authored typically in MS Word are converted to an XML format (OU-XML) and then rendered variously to HTML for the Moodle website, ebook formats, and perhaps PDF (we don’t want to make it too easy for students to print of the materials…).

An internal project that ran for a couple of years (maybe a bit more) looking at more direct authoring workflows was shelved earlier this year. (I was banned from blogging about it whilst it was under development, so I’m afraid I don’t have screen shots to show what it looked like from the time I was given preview access.) As far as I know, the authoring tool was completely distinct from the one developed by the OU’s bastard offspring that is FutureLearn. Nowt like sharing.

One of the things I’m slated to do over the next few months is update, or possibly rewrite, a unit in a first year equivalent module.

My preferred way of authoring for some time has been to keep it simple and just use markdown.

So that’s what I’m probably going to do.

If there’s any griping or sniping that it doesn’t fit the OU workflow, I’ll just run it through pandoc to generate an MS Word docx version and hand that over.

(I’ve been saying *for years* we should have pandoc read/write filters for OU-XML (the most recent notes are here). It would have been a damn site cheaper than the aborted authoring tool project and would have allowed authors to explain a whole range of tools for creating their warez, with pandoc handling the conversion to OU-XML. And yes, I f**king know that some hand cleaning of the OU-XML would almost certainly have been required but we’d have got a far better feeling for what sorts of document structures folk produce if they were allowed to use the tools that suit them. And authors’ shonky mark-up (including my own) *always* needs some fettling anyway: we already know that…)

So… markdown…

If I’m going to revise the current materials, I need to get them out of the current format and into markdown. I’ve previously started looking at an XSLT to convert OU-XML to markdown, eg as described in Fragment – OpenLearn Jupyter Books Remix; a copy of the current-ish XSLT, and some code fragments to grab and convret an example OU-XML document, can be found here.

But today, I thought of an even scruffier and quicker way…

Within the VLE, a single OU-XML source document is rendered across multiple HTML pages, along  with a navigation index:

A single HTML page view (for easier printing) is also available… Hmmm…there are plenty of HTML2markdown converters out there, aren’t there?

#!pip3 install markdownify
from bs4 import BeautifulSoup
from markdownify import markdownify as md

with open('Robotics study week 1 – Introduction_ View as single page.html', 'r') as f:
    # Let's just grab the HTML body...
    tree = BeautifulSoup(, 'lxml')
    body = tree.body
    txt = md(str(body))
with open('','w') as f:
    # There'll still be script tag cruft, videos won't be embedded / linked etc
    # but it's enough to get started with and the diffs should be easy to see...

The output is a bit flakey in parts, but most of the stuff I need is there.  Certainly, there’s more than enough of it in useable form for me to start using as an outline. Indeed, much of the work will be ripping out and replacing the huge chunks of content that are now rather dated.

I can also edit the markdown in a notebook environment using Jupytext, using metadata cells to highlight certain blocks of content with additional structural or semantic metadata, saving the metadata into the markdown document from where it could be processed (I’m not sure how it would turn up if the enhanced markup were converted to docx using pandoc, for example?).

From what I saw of the aborted OpenCreate editor, it used a block/cell style metaphor for creating separate content elements within a page, so it’d also be interesting to compare the jupytext/metadata enhanced markdown, or even the notebook ipynb output format, with the OpenCreate document format / representation to see whether there are similarities in the block level semantic / structural markup.

Nudging Student Coders into Conforming with the PEP8 Python Style Guide Using Jupyter Notebooks, flake8 and pycodestyle_magic

My code is often a mishmash of styles, although I do try to be internally consistent in style in any given notebook or module. And whilst we had the intention that all the code in our TM351 notebooks would be strongly PEP8 compliant, much of it probably isn’t.

So as we start another presentation of TM351, I think that this year I am going to run the risk of adding even more stuff to the student workload in the form of optional, yet regularly posted, notebook productivity tips.

Whilst these will not directly address any of the module learning outcomes that I can recall offhand, they may help students develop their own code in a more efficient way than they might otherwise, and also present it rather more tidily in assessment material. (The latter can often have the effect of improving a marker’s mood, which in turn may influence the mark awarded…)

So what sorts of thing do I intend to cover?

  • simple debugging strategies for one thing: we don’t really teach, or debug, any formal approaches to debugging, although we do encourage “an interactive line at a time” approach to trying out, and developing, data cleaning, shaping, analysis and visualisation code sequences in the notebooks; however, the Python interactive debugger is available in the notebooks too and I think that providing some simple, and relevant, examples of how to use it once student have developed some familiarity with both the notebooks and the style of coding we are using in the course, may be helpful to some of them;

  • simple notebook extensions for monitoring cell execution state on the one hand and profiling code execution on the other, is another area that doesn’t directly address the topic matter directly (coding for data management and analysis), but will provide students with tools that allow them to explore and interrogate their own code in a rather more structured way than they might otherwise;

  • code styling and linting is the first thing I’m going to focus on, however; the intention here is to introduce students to some simple tools and strategies for writing code that conforms to PEP8 style guidelines.

The approach I’m probably going to take is to publish “nudging” posts into the forums once every week or two. Here’s an example of the sort of thing I posted today to introduce the notion of linting and code styling:

Writing Nicely Styled Python Code

In professional code development projects, code it typically written according to a style guide that describes a convention for how to present and layout the code.

In Python projects, the PEP8 style guide defines one such widely followed convention (the code we have provided in the notebooks tends towards PEP8 compliance… Each time we revisit a notebook, we try to tighten it up a bit further!).

Several tools are available for use with Jupyter notebooks that support the creation of PEP8 conformant code. The attached notebook provides instruction on how to install and enable one such tool, pycodestyle_magic, which can provide warnings about when your code style diverges from PEP8 conventions.

The notebook describes how to configure your VM to automatically load pycodestyle_magic and, if required, automatically enable it, in each of your notebooks.

The output of the magic takes the form of a report at the bottom of each code cell identifying any stylistic errors in a particular code cell each time that code cell is run:


alt-text: Example of pink warning message area listing PEP8 style guide contraventions generated via pycodestyle_magic

Each line of the report takes the form:


You can toggle line numbers on and off in a code cell by clicking in the code cell and using the keyboard shortcut: ESC-l

You are not required to install the extension, or even write PEP8 compliant code. However, you may find that it does help make your code readable, and that with practice you soon start to write code that does not raise many PEP8 errors.

(Note that some error reports could do with disabling, such as D100; the extension treats each code cell as a Python module, which is conventionally started with a triple double quoted (sic) comment string (eg """My Module."""). The magic does not currently support ignoring specific errors.)

The notebook itself can be found here: Notebook Code Linting.ipynb.