Category: TrackingJupyter

Several years ago, a Mozilla project announced pyodide, a full Python stack, compiled to WebAssembly / WASM, running in the browser. Earlier this year, pyodide was spun out into its own community governed project (Pyodide Spin Out and 0.17 Release) which means it will now stand or fall based on its usefulness to the community. I’m hopeful this is a positive step, and it’ll be interesting to see how active the project becomes over the next few months.

Since then, a full scipy stack appeared, runnable via pyodide, along with the odd false start (most notably, jyve) at getting a Jupyter server running in the browser. Originally, pyodide had supported its own notebook client (indeed, had been created for it) but that project — iodide — soon languished.

As I haven’t really been Tracking Jupyter since summer last year, there are probably more a few projects ticking along that I missed the earliest of signs of and that have only now come to my attention through occasional mentions on social media that have passed my way.

One of these is jupyterlite (docs), “a JupyterLab distribution that runs entirely in the browser built from the ground-up using JupyterLab components and extensions”. It’s not classic notebook, but it does suggest there’s a running jupyter server available as a WASM component…

So why is this interesting?

To run a Jupyter notebook requires three things:

• a client in the browser;
• a Jupyter server to serve the client and connect it to a kernel process;
• a computing environment to execute code in code cells (the kernel process).

If you access a hosted Jupyter environment, someone else manages the Jupyter server and computing environment for you. If you run notebooks locally, you need at least a Jupyter server, and then you can either connect to a remote kernel or run one locally.

To run a multi-user hosted server, you need to run the Jupyter server, and potentially also manage authentication, persistent storage for users to save their notebooks, and the compute backend to serve the kernel processes. This means you need infrastructure of the hard kind (servers, storage, bandwidth), and you become a provider of insfrastructure of the soft kind (jupyer notebooks as a service).

With Jupyter running in the browser, using something like Jupyterlite, all you need is a web server. Which you’re probably already running. The notebook server now runs in the browser; the kernel now runs in the browser; and the client (JupyterLab) continues to run in the browser, just as it ever did.

In JupyterLite, storage is provided by the local browser storgae, which means you need to work with a single browser. (With many browsers, such as Chrome, now offering browser synchronisation, I wonder if the local storage is synched too? If so, then you can work from any browser you can “log in” to to enable synchronisation services.)

To my mind, this is a huge win. You don’t need to host any compute or storage services to make interactive computing available to your users/students/learners: you just need a webserver. And you don’t even need to run your own: the jupyterlite demo runs using Github pages.

For open education, this means you can make a computing environment available, in the browser, using just a webserver, without the overhead, or security concerns, of running a compute backend capabale of running arbitrary, user submitted code.

So that’s one thing.

For learners running things locally, they just need a simple web server (I think this requires serving: clicking on an HTML document to open it in a browser may hit browser issues that expect content served with a particular MIME-type).

A simple web server is the sort of thing that can be easily packaged and distributed, but it still presents something of an overhead in terms os downloading, installing and then running the service.

Perhaps simpler would be distributing the application as a cross-platform electron app? As far as I know, jupyerlite isn’t (yet?!) packaged that way, but there is at least one demo out there of pyodide bundled inside an electron app: inureyes/pyodide-console (presentation). So it’s not hard to imagine bundling and distributing jupyterlite the same way, although the practicalities may prove fiddly. (Or they may not…)

“So what?”, you may say. “If we’re giving students access to anaconda anyway, what benefit does this bring?” Leaving aside the huge questions I have about using things like Anaconda, not least their lack of generality compared to distributing environments using docker containers, for example, and notwithstanding the ability to be able to provide computing environments purely withing the browser as noted earlier, the availablity of a Jupyter server and Jupyter kernel running in the browser makes other things possible, or at least, allows us to entertain the idea of other applications with a view to seeing if they are realisable.

Hmm… maybe chrome can provide the webserver itself?

So what might those things be? Off the top of my head, and without any serious thought at all, several things come immediately to mind.

Firstly, the possibilty of the cross-platfrom electron distribution (which is essentially an application container wrapping a chrome browser and a simple web server).

Secondly, and nosing around a little, there are already VS Code extensions that seem to be riffing on using jupyterlite too; so if you have access to VS Code in the browser, you could perhaps also install a pyodide jupyter kernel and run notebooks using that in VS Code in the browser. (I’m not sure if you can host VS Code using just a simple web server or it needs a nodejs server app?)

Thirdly, it’s not hard to imagine a route towards making interactive books avaliable and served just via a web browser. For example, a jupyter book UI where rather than  having to hook up to a remote jupyter server to run (editable) code cells from the page using thebelab you could just run the cells against the WASM run kernel in the browser. (What would be required to make a thebelab like javaascript package that would allow a Jupyter Book to connect to a jupyterlite server running from the same browser tab?) It would then be possible publish a fully interactive textbook using just a simple web server and no other dependencies. The only piece missing from that jigsaw would be a Jupyter Book extension to allow you to save edited code cells into browser storage; and maybe also add in some means of adding / editing additional html cells (then at a later date adding support for markdown, perhaps).

The availability of a thebelab like package to connect to an “in page” Jupyter environment also means we can support on demand executable code from any code bearing HTML page, such as content pages with code examples in a VLE web page, and without the need for backend server support.

Finally, institutionally, jupyterlite makes it possible to publish a simple Jupyter environment directly from the VLE as a “simple” html page, with no compute backend/traditional Jupyter hosting requirement on the backend. The compute/storage requirement must be provided by the end user in the form of a recent browser and a computer that can cope with running the WASM enviornment inside it.

Related: Fragment – Jupyter Book Electron App.

A fragmentary note to put a watch on Visual Python, a classic Jupyter notebook extension (note that.. a classic notebook extension) to support visual Python programming:

It’s a bit flaky at the moment — the above screenshot shows multiple previews of the selected function code, and the function preview doesn’t properly render things like the function arguments (nor could I get the function to appear in the list of user defined functions), but it’s early days yet.

At first, a blocker to me in terms of suggesting folk internally have at a look at it right now included the apparant inability to define a variable by visual means (all I wanted to do was set a=1) or clear the notebook cells when I wanted to reflow the visual program into the notebook code cell area.

But then I twigged that rather than trying to create a complete program using the visual tools, a better way of using visualpython might be as a helper to code fragments for me in particular use cases.

In the example below, I created the dataframe manually and then used the editor to create a simple plot command that could be inserted into a notebook code cell. The editor picked up on the dataframe I had defined and used that to prepopulate selection lists in the editor.

If the environment becomes the plaything of devs looking to put complex features into the environment, seeing it as a rich power tool for them (and contra to their beliefs, an increasingly hostile environment to novices as more “powerful” features are added and more visual clutter to the environment to scare the hell of users with things that at irrelevant), then the basic usability required for a teaching and learning environment will be lost if users see it as a tool for creating complete programs visually.

For the developers, it’s all too easy to see how the environment could become as much a toy for adding yet more support for yet more packages that can be demonstrated in the environment but never used (because the power users actually prefer using autocomplete in a “proper IDE”) rather than being simplified for use by novices with very, very, very simple programming demands; (just think of the two, three, four lines of code examples that fill the pages of introductory programming text books).

If folk do want a visual editor for data related programming, wouldn’t they use something like Orange, enso, or the new JupyterLab based orchest?

But if you see the Visual Python editor as a tool at the side that essentially operationalises documentation lookup in a way that helps you create opinionated code fragments, where the opinion is essentially a by prodcut of the code templates that are used to generate code from particular visual UI selections, then I think it could be useful as a support tool for creating code snippets, not as an authoring tool for writing a more complete program or computational analysis.

So what will I be watching for? User uptake (proxied by mentions I see of it), some simple documentation, and perhaps a two minute preview video tour (I’m not willing to spend my time on this right now because I think it needs a bit more time in the oven…). The usability should improve as novices get confused and raise issues with how to perform the most basic of tasks and as the noosphere finds a way to conceptualise the sort of usage patterns and workflows that VisualPython supports best.

My intial reaction was a bit negative — it’s too visually complex already for novices, and some really basic usability issues and operations are either missing or broken, if you see it as an editor for creating complete programs. But if you view it as a code generating documentation support tool that lets you hack together a particular code fragment with visual cues that you might otherwise pick up from from documentation, documentation code examples or simple tutorials, then I think it could be useful.

Hmmm… Another thing to try to get my head round in the context of generative workflow tools…