Fragment – OpenLearn Jupyter Books Remix

Following on from my quandary the other day, I stuck with the simple markdown converter route for OU-XML and now have the first glimmerings of a proof of concept conversion route from OU-XML to Jupyter books.

As an example, I grabbed the OU-XML for the Learn to Code for Data Analysis course on OpenLearn, used XSLT to transform it to markdown, and then published the markdown to Github using Jupyter Book.

There’s an example here (repo).

You’ll notice quite a few issues with it:

• there are no images (the OU-XML uses file paths to images that don’t resolve; there is a way round this — grab links from the HTML version of the course and swap those in for the OU-XML image links — but it’s fiddly; I have code that does it for scrapes of courses from the VLE which will work in the OpenLearn setting, but I should tidy the code so it works nicely for the VLE and OpenLearn so it’s still on the to do list…);
• code is extracted but so are bits of code output, code error messages etc. The way code / code outputs are handled in OU-XML is a bit hit and miss (crap in, crap out etc) but it’s okay to start working from;
• I haven’t done proper XLST handlers for all the OU-XML elements yet; if you see something wrong, let me know and I’ll try to fix it.

The markdown produced by the XLST can be used by Jupytext to generate notebooks (or if you have things set up, just open an .md file in the Jupyter notebook UI and it will open as a notebook). One thing the Jupytext commandline converter doesn’t seem to do is add a (dummy) kernelspec attribute to the generated .ipynb file, so if I try to bulk convert .md files to .ipynb and then use those with Jupyter Book, Jupyter Book throws an error. If I open the notebook against a kernel and save it, it works fine with Jupyter Book.

The reason for converting things to .ipynb format is that Jupyter Book can make those pages interactive. I’ve popped a couple of valid  .ipynb rather than .md pages into the demo to illustrate how it (sort of!) works:

The notebook generated pages are enriched with a notebook download button and a ThebeLab button. Clicking the ThebeLab button makes the code cells interactive via ThebeLab/MyBinder.

Unfortunately, most of the code is a bit borked (eg missing dependencies, or missing local files) but you can still edit the code in a ThebeLab’d cell and execute it.

One other issue is that Jupyter Book fires up a separate MyBinder kernel for each notebook / page, which means you can’t carry state between pages. It’d be quite handy if all the ThebeLab panels were connected to the same IPython instance. (@mdpacer has an example of a related sort of common state conversation between a console and a notebook here.) That way, you could run examples in one chapter and carry the state through to later chapters. (Yes, I KNOW there are issues doing that. But if I didn’t raise it, folk would complain that you CAN’T carry state across pages…!)

So… what next?

For content in markdown format, I think you can just edit the files in the book’s _build folder and the book page should be updated automatically (erm, I should check this…). If you enable Jupytext to dual notebooks and markdown, if you edit one of the notebook files in a Jupyter notebook environment, it should create a .md file that could be added to the _build directory and thence update the book page.

In part, the above all speaks to things like Fragment – ROER: Reproducible Open Educational Resources and OERs in Practice: Re-use With Modification where we take openly licensed educational materials and get them into a format that is raw enough for us to be able to edit them relatively easily. We can also get them into an environment where we can start repurposing and modifying them (for example, we could run and edit the notebooks in MyBinder) as well as publishing any modifications we make simply by recommitting any updated files to the Github repo.

The above approach also speaks to Build Your Own Learning Tools (BYOT). For example, now I have a way to relatively easily dump an OpenLearn course into an environment where I can start swapping static bits of content out for interactive bits I may want to develop in their place.

PS For anyone in the OU, I can grab OU course materials that have an OU-XML basis from the VLE, so if there’s a course you’d like me to try it with, let me know…