Fragment: Loading Data into pandas DataFrames in JupyerLite

Just a quick note to self about a couple of tricks for loading data files into pandas in JupyterLite. At the moment, a simple pandas.read_csv() is unlikely to work, but there a couple of workarounds have been posted over the last couple of months so I’ve wrapped them into a simple package until such a time as everything works “properly” – innovationOUtside/ouseful_jupyterlite_utils.

Install the package in jupyterlite as:

import micropip

package_url = "https://raw.githubusercontent.com/innovationOUtside/ouseful_jupyterlite_utils/main/ouseful_jupyterlite_utils-0.0.1-py3-none-any.whl"

await micropip.install(package_url)

And then load data into pandas as per:

from ouseful_jupyterlite_utils import pandas_utils as pdu

# Load CSV from URL
# Via @jtpio
URL = "https://support.staffbase.com/hc/en-us/article_attachments/360009197031/username.csv"
df = await pdu.read_csv_url(URL, "\t")

# Load CSV from local browser storage
# Via @bollwyvl
df = await pdu.read_csv_local("iris.csv", "\t")
df

I’ll add more workarouds to the package as I find them (working with SQLite files is next on my to do list…) and then remove items as they (hopefully) become natively supported.

Preparing Jupyter Notebooks for Release to Students

Over the years, I’ve sketched various tools to support the release of notebooks to students, but as I’m not the person who prepares and distributes the releases, they never get used (“Tony hacking crap again” etc.;-).

Anyway, on the basis that the tools aren’t completely crap, and may be of use to others, perhaps even folk working on other modules internally that make use of notebooks and are using them for the first time this presentation, I’ll post a quick summary of some of them here. (And if they are broken, a little use and testing by not-me could well provide the bug reports and motivation I need to fix them to a level of slightly less possible brokenness.)

The package that bundles the tools can be found here: innovationOUtside/nb_workflow_tools.

First up, tm351nbtest is a tool that helps check whether the notebooks run correctly in the latest environment.

The notebooks we save to the private module team repo all have their cells run, in part so that we can review what the expected outputs are. (When checking in notebooks, the tm351nbrun --file-processor runWithErrors . command can be used to ensure all noebooks in the specified path have their cells run.) The nbval package is a handy package that runs the notebooks in the current environment and checks that the contents of the new output cell match those of the previous, saved output cell. (I keep thinking that jupyter-cache might also be handy here?) Cells that are known to generated an error can be ignored by tagging them with the raises-exception tag and cells you want to ignore the output of can be tagged with the nbval-ignore-output tag. Running the tool generates a report identifying each notebook and each cell where the outputs don’t match.

The next tool, nb_collapse_activities, checks that out activity blocks all have their answers precollapsed. Activities are tagged and coloured using the innovationOUtside/nb_extension_empinken extension; activities with answers use the classic notebook collapsible headings extension to collapse the cells beneath an activity answer heading block (all cells are collapsed to the the cell with a header at the same level or higher as the collapsed answer cell header). The nb_collapse_activities utility tries to identify answer head cells and whenever it finds one, adds heading_collapsed: true metadata.

The third tool also processes the notebooks for release: tm351nbrun --file-processor clearOutput clears the outputs of every code cell and essentially resets each notebook to an unrun state.

A fourth tool, nbzip, can be used to zip required notebook folders for release to students.

A sort of release process could then work soemthing like this. In the environment you want to test in:

# Install package
pip3 install --upgrade git+https://github.com/innovationOUtside/nb_workflow_tools

# When checking in notebooks, ensure cells are run
# Ensure that all cells are run even in presence of errors
tm351nbrun --file-processor runWithErrors .

# Test notebooks
tm351nbtest .

# Quality reports
## Whatever...

# Clear outputs
tm351nbrun --file-processor clearOutput .

# Collapse acvitity answers
nb_collapse_activities .

# Spell check
## However... Or run earlier before output cells cleared

# Zip files
# Whichever...

In passing, the nb_workflow_tools package also includes some other utilities not directly relevant to release, but occasionally handy during production: nb_merge to merge two or more notebooks, and nb_split to split a notebook into to or more notebooks.

I’ve also been exploring various approaches to spell-checking notebooks. These are currently being collected in innovationOUtside/nb_spellchecker and the various issues attached to that repo. When I have something reliable, I’ll add it to innovationOUtside/nb_workflow_tools. Another set of quality tools I had been working on but halted due to a universal “why would we want to know anything comparative about the contents of our notebooks” can be found in innovationOUtside/nb_quality_profile. At some point I’ll revisit this and then try to bundle them up into a simple CLI tool I can also add to nb_workflow_tools.

In passing, and for completeness, I’ve also started sketching some innovationOUtside/ou-jupyter-book-tools. The idea of these is that they can provide an intermediate publishing step, where necessary, that maps from cell tags, for example, to complementary Jupyter Book / MyST / restructured text markdown.

Helping Students Make Sense of Code Execution and Their Own Broken Code

The programming related courses I work on are probably best described as introductory programming courses. Students are taught using an approach based on “a line of code at a time” within a Jupyter notebook environment which provides a REPL execution model. Students are encouraged to write a line of code in a cell, run it, and then inspect state changes arising from the code execution as displayed in the code cell output. Markdown cells before an after the code cell are use to explain the motivation for the next bit of code, or prompting students to predict what they think it will do. Markdown cells following a code cell can be use to review or explain what just happened, or prompt students to reflect on what they think happened.

In passing, I note that there are other models for providing text+code style annotations. For example the pycco-docs/pycco package will render side-by-side comments and code:

pycco side by side literate documentation, generated from comment py code

The view is generated from Python files containing inline comments and docstrings:

Source py file from which pycco side-by-side literate code view is generated

Something I haven’t yet tried is a workflow that renders the side-by-side view from a Python file generated from a Jupyter notebook using the jupytext file converter (I’m not sure if Jupytext can generate the python files using comment markup conventions that pycco expects?)

For simple code blocks, tools such as nbtutor provide a simple code stepper and tracer that can be used to explore the behaviour of a few lines of code.

I use nbtutor in some first year undergraduate notebooks and it’s okay, -ish (unfortunately it can break in combination with some other widgets running in the same notebook).

Another approach I am keen to explore in terms of helping students help themselves when it comes to understanding code they have written is the automated generation of simple flowchart visualisations from code fragements (see for example Helping Learners Look at Their Code).

Poking around looking for various Python packages that can help animate or visualise common algorithms (Bjarten/alvito is one; anyone got suggestions for others?) I came across a couple of other code stepping tools produced by Alex Hall / @alexmojaki.

The first one is alexmojaki/birdseye which can provide a step trace for code executed in a magicked notebook cell block:

COde stepper using birdseye

You can also separately step though nested loops:

Step through nested loops using birdseye

Another tool, alexmojaki/snoop, will give a linear trace from an executed code cell:

Linear code trace using snoop

Alex also has a handy package for helping identify out of data Python packages based on the latest version availble on PyPi: alexmojaki/outdated.

When it comes to Python errors, for years we have used the Jupyter skip-traceback extension to minimise the traceback message displayed when an error is raised in a Jupyter notebook. However, there are various packages out there that attempt to provide more helpful error messages, such as SylvainDe/DidYouMean-Python (which is currently broken from the install – I think the package needs its internal paths fettling a bit!) and friendly-traceback. The latter package tidies up the display of error messages:

Simplified error message using friendly-traceback

Note that the pink gutter to indicate failed cell execution comes from the innovationOUtside/nb_cell_execution_status extension.

You can then explore in more detail what the issue is and in some cases, how you might be able to fix it:

friendly-traceback error detail

You can also start to tunnel down for more detail about the error:

friendly-traceback messages and explanations

This extension looks like it could be really handy in an introductory, first year undergraduate intro to programming module, but the aesthetic may be a bit simplistic for higher level courses.

From the repo, friendly-traceback/friendly-traceback, it looks like it shouldn’t be too hard to create your own messages.

friendly-traceback feedback generation

This does make me wonder whether a language pack approach might be useful? That would allow for internationalisation but could also be used to easily support the maintenance of custom message packs for particular teaching and learning use cases?

With a couple of new modules presenting for the first time this year, I would argue we missed an opportunity to explore some of these ideas where we can start to use the technology as an illustrator of what’s going on with code we give to students, and more importantly that students might write for themselves.

There are several reasons why I think this probably hasn’t happened:

  • no time to explore this sort of thing (with two years+ to produce a course, you might want to debate that…);
  • no capacity in a module team to explore and test new approaches (I’d argue that’s our job as much as producing teaching material if the org is a beacon of best practice in the development and delivery of interactive online distance education materials);
  • no capacity in support units to either research their effectiveness or explore such approaches and make recommendations into module teams about how they might be adopted and used, along with an examples gallery and sample worked examples based on current draft materials (I honestly wonder about where all the value add we used to get from support units years ago has gone and why folk don’t think we are the worse for not having units that explore emerging tech for teaching and learning. Folk too busy doing crapalytics and bollockschain, I guess);
  • and I guess: “what value does it add anyway?” (which is to say: “why should we explore new ways of teaching and learning?”) and “you’re just chasing ooh, shiny” (which really doesn’t fit with 2+ year production cycles and material updates every five years where locking into an emerging technology is high risk, becuase rather than regularly updating around it, you are stuck with it for potentially up to a decade (2 years production, five years primary course life, 3 years course life extension)).

Bored, bored, bored, bored, bored…

Simple Javascript Multiple Choice And Literal Answer Quizzes in Jupyter Notebooks and Jupyter Book

In Helping Students Make Sense of Code Execution and Their Own Broken Code I described a handful of interactive tools to help students have a conversation with a code fragment about what it was doing. In this notebook, I’ll consider another form of interactivity that we can bring to bear in live notebooks as well as static Jupyter Book outputs: simple multiple choice and literal answer quizzes (which I like) and flashcards (which I’ve never really got on with).

Both examples are created by John Shea / @jmshea, whose Intro to Data Science for Engineers Jupyter Book demonstrates some interesting practice.

Keen observers of this blog will note I don’t tend to link to demos of OU materials (only my own that I have drafted in public). That’s because it’s generally authed and only available to paying students. Unlike OU print materials of yore, which could be found in many College libraries, purchased as standalone study packs, or bought second hand from University Book Search. Some materials are on Open Learn, and I keep meaning to give some of them “a treatment” to show other, and to my mind, more engaging, ways in which I think we could present them… When the next strike comes around, maybe…

The jmshea/jupyterquiz package provides support for a range of simple quiz questions that can be used for untracked formative assessment.

Try a Jupyter Book rendering here and a previewed Jupyter notebook rendering here.

The first question type currently supported is a multiple choice question type where a single correct answer is expected (a "multiple_choice" question type).

jupyter-quiz – multiple choice

Hints can also be provided in the event of an incorrect answer being provided:

jupyter-quiz – multiple choice with hint on incorrect answer

The second, related "many_choice" question type requires the user to select M correct correct answers from N choices.

The third quiz type, "numeric" allows a user to check a literal numeric answer and provide a hint if the incorrect answer is provided:

jupyter-quiz – literal answer test with hint on incorrect answer

It strikes me it should be trivial to add an exact string match test, and if Javascript packages are available, simple fuzzy string match tests etc.

The questions and answers can be pulled in from a JSON file hosted locally or retrieved from a URL, or a Python dictionary.

Here’s an example of a "multiple_choice" question type:

{
        "question": "Which of these are used to create formatted text in Jupyter notebooks?",
        "type": "multiple_choice",
        "answers": [
            {
                "answer": "Wiki markup",
                "correct": false,
                "feedback": "False."
            },
            {
                "answer": "SVG",
                "correct": false,
                "feedback": "False."
            },
            {
                "answer": "Markdown",
                "correct": true,
                "feedback": "Correct."
            },
            {
                "answer": "Rich Text",
                "correct": false,
                "feedback": "False."
            }
        ]
    },

And a numeric quiz type:

{
        "question": "The variable mylist is a Python list. Choose which code snippet will append the item 3 to mylist.",
        "type": "multiple_choice",
        "answers": [
            {
                "code": "mylist+=3",
                "correct": false
            },
            {
                "code": "mylist+=[3]",
                "correct": true
            },
            {
                "code": "mylist+={3}",
                "correct": false
            }
        ]
    },

The second package, jmshea/jupytercards, supports the embedding of interactive flash cards in Jupyter notebooks and Jupyter Book.

Clicking on the flashcard turns it to show the other side:

You can also transition from one flashcard to the next:

The flashcards can be loaded from a local or remotely hosted JSON text file listing each of the flash card texts in a simple dictionary:

[
    {
        "front": "outcome (of a random experiment)",
        "back": "An outcome of a random experiment is a result of the experiment that cannot be further decomposed."
    },
    {
        "front": "sample space",
        "back": "The sample space of a random experiment is the set of all possible outcomes."
    },
    {
        "front": "event class",
        "back": "For a sample space $S$ and a probability measure $P$, the event class, denoted by $\\mathcal{F}$, is a collection of all subsets of $S$ to which we will assign probability (i.e., for which $P$ will be defined). The sets in $\\mathcal{F}$ are called events."
    }
]

I’m not sure if you can control the flash card color, or font style, color and size?

What I quite like about these activities is that they slot in neatly into generative workflows: the questions are easily maintained via a source text file or a hidden cell where the JSON data is loaded into a Python dict (I suppose it could even be pulled in from notebook cell metadata) and can then be used in a live (trusted) notebook, a fully rendered notebook (i.e. one rendered by nbviewer, not the Github notebook previweer) or rendered into a Jupyter Book HTML format.

Note to self: add these examples to my Open Jupyter Authoring and Learning Environment (OpenJALE) online HTML book.

Simple Markdown Table Output From pandas DataFrames

In passing, I note that we can easily generate simple markdown style table output from a pandas data frame using the pandas .to_markdown() dataframe method:

Here’s the associated code fragment:

data = """colA, colB, colC
this, that, 1
or, another, 2"""

import pandas as pd
from io import StringIO

df = pd.read_csv(StringIO(data))
md_table = df.to_markdown(index=False)

print(md_table)

I also note that Stack Overflow recently (ish!) added support for markdown tables (announcement) using Github-flavoured markdown syntax.

Getting nbev3devsim (jp_proxy_widget wrapped Javascript App) Running In JupyterLab With JupyterLab-Sidecar

One of the blockers I’ve had to date running the nbev3devsim simulator in JupyterLab was the requirement to have just a single instance of the widget running in a notebook. This is an artefact (I think) of the jp_proxy_widget having a hardwired HTML element reference that needs to be unique.

The JupyterLab sidecar widget offered promise in the form of mirroring a cell output in a separate JupyterLab panel, the only problem being this then created a second widget instance (once in the original cell output, once in a mirroring panel).

But it seems that a (not so?) recent update to the sidecar extension allows you to display a rendered object not in a JuptyerLab cell output, but instead in a separate, detached panel:

nbev3devsim Simulator running in a JupyterLab sidecar widget

This compares to the current classic notebook view where I style the css to dodge the notebook to the left hand side of the page and then pop the simulator into a floating resizable widget:

For a review of nbev3devsim and the materials associated with its use, see this overview.

The recipe for getting the widget displayed in the JupyterLab sidecar panel is as follows:

#%pip install git+https://github.com/innovationOUtside/nbev3devsim.git
#%pip install sidecar
#%pip install jp_proxy_widget
# !jupyter labextension install jp_proxy_widget
# Reload...

from nbev3devsim import ev3devsim_nb as eds

roboSim = eds.Ev3DevWidget()

roboSim.set_element("response", '')

from sidecar import Sidecar

with Sidecar(title='RobotLab', anchor='split-right'):
    display(roboSim)

This seems to work fine on the jp_proxy_widget Binder server into which the sidecar package is installed (and the environment reloaded) but there’s a jp_proxy_widget error (even with a simple test widget) when installing the jp_proxy_widget package into the Binderised sidecar repo. So there’s presumably some package version conflict somewhere? Works fine if you make sure the proxy widget extension is explicitly installed.

In the jp_proxy_widget Binder environment, the plumbing also seems to work: loading in the magic (%load_ext nbev3devsim) and then running via the block magic (`%%sim_magic` or %%sim_magic -s roboSim) and programs seem to run in the simulator okay. Line magics to control the simulator setup also seem to work okay.

There may be a few things, particularly to do with accessibility, that work (ish?!) in the classic notebook UI that break in the JupyterLab UI. A move to the JupyerLab UI also means that icons for launching server proxy apps from the launcher will be required.

But it’s starting to look that the robot simulator is viable in JupyterLab. I just need to find a way to get the notebook customisation extensions such as nb_extension_empinken and nb_extension_tagstyler ported over.

Electron Powered Desktop Apps That Bundle Python — Datasette and JupyterLab

You wait for what feels like years, then couple of in the wild releases appear that put the same pattern into practice within a few weeks of each other!

A couple of years ago, whilst pondering ways of bundling Jupyter Book content in a desktop electron app to try to get round the need for separate webserver to serve the content, I observed:

Looking around, a lot of electron apps seem to require the additional installation of Python environments on host that are then called from the electron app. Finding a robust recipe for bundling Python environments within electron apps would be really useful I think?

Fragment – Jupyter Book Electron App, ouseful.info, May, 2019

And suddenly, in a couple of the projects I track, this now seems to be A Thing.

For example, datasette-app (Mac only at the moment?) bundles a Python environment and a datasette server in an electron app to give you a handy cross-desktop application for playing with datasette.

I need to do a proper review of this app…

Simon Willison’s extensive developer notes, tracked in Github Issues and PRs, tell the tale. For example:

And from a repo that appeared to have gone stale, jupyterlab-app gets an initial release as an electon app bundling an Anaconda environment with some handy packages pre-installed (announcement; cross-platform (Mac, Linux, Windows)).

Naively double-clicking the downloaded JupyterLab-app installer to open raises a not helpful dialog:

Sigining issue with Jupyterlab app on Mac

To make progress, you need to list the app in a Mac Finder window, right-click it and Open, at which point you get a dialog that is a little more helpful:

The download for the JupyterLab-app (Mac) installer was about 300MB, which expands to an installed size of at least double that:

The installation (which requires an admin user password?) takes some time so I’m wondering if load of other things get downloaded and installed as part of the installation process….

Hmmm… on a first run, the app opens up some notebooks that I think I may have been running in another JupyterLab session from a local server – has it actually picked up that JupyterLab workspace context? The announcement post said it shipped with its own conda environment? So where has it picked up a directory path from?

Hmmm… it seems to have found my other kernels too… But I don’t see a new one for the environment, and kernel, it ships with?

Opening the Python 3 (ipykernel) environment appears to give me a kernel that has shipped with the application:

import site
site.getsitepackages()

I wonder if there is a name clash with a pre-existing external kernel, the kernel that ships with the JupyterLab app it uses it’s local one, otherwise it uses the other kernels it can find?

Hmmm… seems like trying to run the other Python kernels gets stuck trying to connect and then freezes the app. But I can connect to the R kernel, which looks like it’s the “external” kernel based on where it thinks the packages are installed (retrieved via the R .libPaths() lookup):

Something else that might be handy would be the ability to connect to a remote server and launch a remote kernel, or launch and connect to a MyBinder kernel…

I also note that if I open several notebooks in the app, then in my browser launch a new JupyterLab session, the open notebooks in the app appear as open notebooks in the browser JupyterLab UI: so workspaces are shared? That really really sucks. Sucks double. Not only are the workspaces messing each other up, but it means not only that the same notebook is open in two environments (causing write conflicts) but those environments are also using different kernels for the same notebook. Wrong. Wrong. Wrong!;-)

But a handy start… and possibly a useful way of shipping simple environments to students, at least, once the signing issues are fixed. (For a related discussion on signing Mac apps, see @simonw’s TIL tale, “Signing and notarizing an Electron app for distribution using GitHub Actions” and this related JupyterLab-app issue.)

I also wonder: as the electron app bundles conda, could it also ship postgres as a callable database inside the app? Postgres is available via conda, after all..

Hmm… thinks… did I also see somewhere discussion about a possible JuypterLite app that would use WASM kernels rather than eg bundling conda or accessing local Jupyyter servers? And I wonder… how much of the JupyterLab-App repo could be lifted and used to directly wrap RetroLab? Or maybe, the JupyterLab app could have a button to switch to RetroLab view, and vice versa?

Hmm… I wonder… the jupyter-server/jupyter_releaser package looks to be providing support for automating builds and release generators for templated Jupyter projects… How tightly are apps bound into electron packages? How easy would it be to have a releaser to wrap an app in an electron package with a specfied conda distribution (an “electron-py-app proxy” into which you could drop your jupyter framework app?)

PS whilst tinkering with the fragment describing a Jupyter Book Electron App, I wrapped a Jupyter book in an electron app to remove the need for a web server to serve the book. That post also briefly also explored the possibility of providing live code execution via a thebe connected local server, as well as mooting the possibilty of executing the code via pyodide. I wonder if the JupyterLab-app, and or the datasette-app, have elecron+py components cleanly separated from the app code components. How easy would it be to take one or other of those components to create an electron app bundled with a py/jupyter server that could support live code execution from the Jupyer Book also inside the electron app?

Towards One (or More!) Institutionally Hosted Jupyter Services

When we first presented a module to students that used Jupyter notebooks (which is to say, “IPython notebooks” as they were still referred to back in early 2016) we were keen that students should also be able to access an online hosted solution as an alternative. Last year, we provided an on-request hosted service for the data management module, but it was far from ideal, running as it did an environment that differed from the environment we provided to students via a Docker container.

This year, we are offering a more complete hosted service for the same module that makes use of pretty much the same Dockerised environment that the students can run locally (the difference is that the hosted solution also bundles the JupyterHub package and a shim to provide a slightly different start-up sequence).

The module is hosted by the schools specialist IT service using a customised zero2kubernetes deployment route developed by an academic colleague, Mark Hall, as part of an internal scholarly research project exploring the effectiveness of using a hosted environment for a web technology course. (Note that: the developer is an academic…)

The student user route is to click on an LTI auth link in the Moodle VLE that takes ths students to an “available images” list:

On starting an image, an animated splash screen shows the deployment progress. This can take a couple of minutes, depending on the deployment policy used (e.g. whether a new server needs to be started or whether the container can be squeezed onto an already running server.

Cost considerations play a part here in determining resource availability, although the data management module runs on local servers rather than commercial cloud servers.)

Once the environment has started up and been allocated, a startup sequence checks that everything is in place. The container is mounted against a persistent user filespace and can deploy files stashed inside the container to that mounted volume on first run.

Once again, there may be some delay as any required files are copied over and the actual user environment services are started up:

When the user environment is available, for example, a Jupyter notebook environment, the student is automatically redirected to it.

The Control Panel link takes the student back to the server image selection page.

In terms of user permissions, the containers are intended to run the user as a permission limited generic user (i.e. not root). This has some downsides, not least in terms of slightly increasing the complexity of the environment to ensure that permissions are appropriately set on the user. Also note that the user is not a unique user (e.g. all users might be user oustudent inside their own container, rather than being their own user as set using their login ID).

The Docker image used for the data management course is an updated version of the image used for October 2020.

A second module I work on is using the same deployment framework but hosted by the academic developer using Azure servers. The image for that environment is an opinionated one constructed using a templated image building tool, mmh352/ou-container-builder. The use of this tool is intended to simplify image creation and management. For example, the same persistent user file volume is used for all launched computational environments, so care needs to be taken that an environment used for one course doesn’t clobber files or environment settings used by another course in the persistent file volume.

One thing that we haven’t yet bundled in our Jupyter containers is a file browser; but if students are mounting persisting files against different images, and do want to be able to easily upload and download files, I wonder if it makes sense to start adding such a service vie a Jupyter server proxy wrapper. For example, imjoy-team/imjoy-elfinder [repo] provides such a proxy off-the-shelf:

The above approach, albeit variously hosted and resourced, uses JupyterHub to manage the (containerised) user environments. A second approach has been taken by central IT in their development of a generic Docker container launch.

I note that you can use JupyterHub to manage arbitrary services running in containers, not just Jupyter notebook server environments, but from what I can tell, I don’t think that central IT even evaluated that route.

As with the previous approach, the student user route in is via an LTI authentication link in the browser. The landing page sets up the expectation of a long wait…

Telling a student who may be grabbing a quick 30 minute study session over lunch that they must wait up to 10 minutes for the environment to appear is not ideal… Multiply that wait by 1000 students on a course, maybe two or three times a week for 20+ weeks, and that is a lot of lost study time… But then, as far as IT goes, I guess you cost it as “externality”…

To add to the pain, different containers are used for different parts of the module, or at least, different containers are used for teaching and assessment. Since a student can only run one container at a time, if you start the wrong cotnainer (and wait 5+ minutes for it to start up) and then half to shut it down to start the cotnainer you meant to start), I imagine this could be very frustrating…

As well as the different Jupyter environment containers, a file manager container is also provided (I think this can be run at the same time as one of the Jupyer container images). Rather than providing a container image selection UI on an integrated image launcher page, separate image launching links are provided (and need to be maintained) in the VLE:

The file browser service is a containerised version of TinyFileManager [repo]:

In the (unbranded) Jupyter containers, the environment is password protected using a common Jupyter token:

The (unbranded) environment is wrapped in an IFrame by default:

However, if you click on the Jupyter link you can get a link to the notebook homepage without the IFrame wrapper:

In passing, I note that in this IT maintained image, the JupyterLab UI is not installed, which means students are required to use the notebook UI.

It’s also worth noting that these containers are running on GPU enabled servers. Students are not provided with a means of running environments locally because some of the activities require a GPU if they are to complete in a timely fashion.

In passing, I note that another new module that started this year that also used Jupyter notebooks does not offer a hosted solution but instead instructs students to download and install Anaconda and run a Jupyter server that way. This module has very simple Python requirements and I suspect that most, if not all, the code related activities could be executed using a JupyterLite/Pyodide kernel run via WASM in the browser. In many cases, the presentation format of the materials (a code text book, essentially) are also suggestive of a Jupyter Book+Thebe code execution model to me, although the the ability for students to save any edited code to browser storage, for example, would probably be required [related issue].

One final comment to make about the hosted solutions is that the way they are accessed via an LTI authenticated link using institutional credentials prevents a simple connection to the Jupyter server as a remote kernel provider. For example, it is not possibly to trivially launch a remote kernel on the hosted server via a locally running VS Code environment. In the case of the GPU servers, this would be really useful because it would allow students to run local servers most of the time and then only access a GPU powered server when required for a particular activity.

Fragment: Basic Computing Concepts for Students – File Paths

A widely shared blog post – File not found – has been doing the rounds lately that describes how for a large proportion of students, the question of “where” a file is on their computer is a meaningless question.

We see this from time to time in a couple of modules I work on, even amongst third year undergrad students, where the problem of not being able to locate a file manifests itself when students have to open files from a code by referring to the file’s address or file path. “Path? Wossat then?”

Orienting yourself to the current working directory (“the what?”) and then using either relative or absolute addressing (“either what or what?”) on the filesystem (“the f…?”) to move or view the contents of another directory are just not on the radar…

In the courses I work on, the problems are compounded by having two file systems in place, one for the students desktop, one inside a local docker container; some directories mounted from host onto a path inside the virtual environment as a shared folder. (If none of that makes sense, but you laughed at the thought of students not being able to reference file locations, then in the world we now live in: that… And: you…)

I used to think that one of the ways in to giving students some sort of idea about what file paths are was to get them to hack URLs, but I could never convince any of my colleagues that was a sensible approach.

I still think they’re wrong, but I’m not sure I’m right any more… I’ve certainly stopped hacking URLs so much myself in recent years, so I figure that URL patterns are perhaps changing for the worse. Or I’m just not having to hack my own way around websites so much any more?

Anyway… the blog post immediately brought to mind a book that was championed folksonomies, back in the day, a decade or so ago, when tagging was all the rage amongst the cool library kids, and if you knew what faceted browsing was and knew how to wield a ctrl-f, you were ahead of the curve. (I suspect many folk have no idea what any of that meant…). The book was, David Weinberger’s Everything is Miscellaneous, and pretty much every one who was into that sort of thing would have read it, or heard one of his presentations about it:

I would have linked to it on Hive or bookshop.org.uk, but a quick naive search on those sites (which are essentially the same site in terms of catalogue, I think?) didn’t find the book…

Anyway, I think getting students to read that book, or watch something like embedded video above, where David Weinberger does the book talk in a Talks at Google session from a hundred years ago, might help…

See also: Anatomy for First Year Computing and IT Students, another of those crazy ideas that I could never persuade any of my colleagues about.

Pondering: Stories Not Accounts

Chatting with Island Storytellers convener Sue Bailey last week, I commented that I really need to put more time into getting the end of the stories I try to tell much tighter. If you have a good opening, and a really strong ending, then you can generally get from one to the other. But if the ending isn’t as solid as it should be, you start to worry about how to close as you get closer to it, and may even finish without anyone realising.

A good title can also help, and often captures a key scene or storypoint from somewhere in the middle.

We also chatted about sourcing stories from local histories. I’ve started trying to pull together a small set of Island history stories I can tell, and I’ve also got a longer tale about the Yorkshire Luddites; but as much as anything, they’re told as accounts rather than stories (thanks to Sue for introducing the distinction of an account into the discussion).

So pondering a handful of stories I’ve not quite got round to pulling together yet into a form that I can tell, I think I’m going to have a go at quickly summarising first the account, which will give me lots of facts and footnotes and depth behind the tale, and then try to to turn them into stories by taking a particular perspective – a particular person, or place, or animal, for example – whose journey we can follow (because a good story is often about something or someone).

For my own reference (so I can keep track of progress, as much as anything), the tales I’ll work on are:

  • the wrecking of the St Mary and the building of The Pepperpot by Walter de Godeton; this could be told from de Godeton’s perpective, but it might also be interesting to try it as a story about a barrel of communion wine, or the Abbey that was expecting the wine…; [story and account]
  • when the Island invaded France, a tale of Sir Edward Woodville (probably; or maybe a story about Diccon Cheke, the sole returning survivor…);
  • The Worsley trial: it could be interesting to be able to tell this two ways: as a story about Richard Worsley, or a story about Seymour Fleming;
  • Odo’s gold, perhaps from the perspective of Odo, perhaps from the perspective of Carisbrooke Castle…

I’m also going to have a go at recasting my Yorkshire Luddite account, I think as a story about George Mellor…