TJ Fragment: Sharing Desktop Apps Via Jupyter-Server-Proxy Et Al.

It’s been some time since I last had a play with remote desktops, so here’s a placeholder / round up of a couple of related Jupyter server proxy extensions that seem to fit the bill.

For those at the back who aren’t keeping up, jupyter-server-proxy applications are incredibly useful: they extent the Jupyter server to proxy other services running in the same environment. So if you have a Jupyter server running on example.url/nbserver/, and another application that publishes a web UI in the same environment, you can publish that application, using jupyter-server-proxy, via example.url/myapplication. As an example, for out TM351 Data Management and Analysis course, we proxy OpenRefine using jupyter-server-proxy (example [still missing docs].).

Applications that are published using a jupyter-server-proxy wrapper are typically applications that publish an HTML UI. So what do you do if the application you want to share is a desktop application? One way is to to share the desktop via a browser (HTML) interface. Two popular ways of doing this are:

  • novnc: an “open source VNC client – it’s is both a VNC client JavaScript library as well as an application built on top of that library”;
  • xpra: “an open-source multi-platform persistent remote display server and client for forwarding applications and desktop screens”.

Both of these applications allow you to share (Linux) desktop applications via a web browser, and both of them are available as jupyter-server-proxy extensions (subject to the correct operating system packages also being installed).

As far as novnc goes, jupyterhub/jupyter-remote-desktop-proxy will “run a Linux desktop on the Jupyter single-user server, and proxy it to your browser using VNC via Jupyter”. A TightVNC server is bundled with the application as a fallback if no other VNC server is available. One popular application used wrapped by several people using jupyter-remote-desktop-proxy is QGIS; for example, giswqs/jupyter-qgis. I used it to demonstrate how we could make a legacy Windows desktop application available via a browser by running using Wine on a Linux desktop and then sharing it via the jupyter-remote-desktop-proxy.

For xpra, the not very active (but maybe it’s stable enough?!) FZJ-JSC/jupyter-xprahtml5-proxy seems to allow you to “integrate Xpra in your Jupyter environment for an fast, feature-rich and easy to use remote desktop in the browser”. However, no MyBinder demo is provided and I haven’t had a chance yet to give this a go. (I have tried XPRA in other contexts, though, such as here: Running Legacy Windows Desktop Applications Under Wine Directly in the Browser Via XPRA Containers.)

Another way of sharing desktops is to use the Microsoft Remote Desktop Protocol (aka RDP). Again, I’ve used that in various demos (eg This is What I Keep Trying to Say…) but not via a jupyter-server-proxy. I’m not sure if there is a jupyter-server-proxy example out there for publishing a proxied RDP port?

Just in passing, I also note this recipe for a Docker compose configuration that uses a bespoke container to act as a desktop sharing bridge: Viewing Dockerised Desktops via an X11 Bridge, novnc and RDP, Sort of…. I’m not sure how that might fit into in Jupyter set up? Could a Jupyter server container be composed with a bridge container, and then proxy the bridge services?

Finally, another way to share stuff is to to use WebRTC. The maartenbreddels/ipywebrtc extension can “expose the WebRTC and MediaStream API in a Jupyter notebook/JupyterLab environment” allowing you to create a MediaStream out of an ipywidget, a video/image/audio file, or a webcam and use it as the bases for a movie, image snapshot or audio recording. I keep thinking this might be really useful for recording screencast scenes or other teaching related assets, but I haven’t fully grocked the full use of it. (Something like Jupyter Graffiti also falls into this class, which can be used to record a “tour” or walkthrough of a notebook that can also be interrupted by live interaction or the user going off-piste. The jupyterlab-contrib/jupyterlab-tour extension also provides an example of a traditional UI tour for JupyterLab, although I’m not sure how easy it is to script/create your own tours. Such a thing might be useful for guiding a user around a custom JupyterLab workspace layout, for example. [To my mind, workspaces are the most useful and least talked about feature of the JupyerLab UI….] More generally, shepherd.js looks interesting as a generic website tour supporting Javascript package.) What I’m not sure about is the extent to which I could share, or proxy access to, of a WebRTC MediaStream that could be accessed live by a remote user.

Another way of sharing the content of a live notebook is to use the new realtime collaboration features in JupyterLab (see the official announcement/background post: How we made Jupyter Notebooks collaborative with Yjs). (A handy spin-off of this is that it now provides a hacky workaround way of opening two notebooks on different monitors.) If you prefer more literal screensharing, there’s also yuvipanda/jupyter-videochat which provides a server extension for proxying a Jitsi (WebRTC) powered video chat, which can also support screen sharing.

jupyterlite — “serverless” Jupyter In the Browser Using Pyodide and WASM

Several years ago, a Mozilla project announced pyodide, a full Python stack, compiled to WebAssembly / WASM, running in the browser. Earlier this year, pyodide was spun out into its own community governed project (Pyodide Spin Out and 0.17 Release) which means it will now stand or fall based on its usefulness to the community. I’m hopeful this is a positive step, and it’ll be interesting to see how active the project becomes over the next few months.

Since then, a full scipy stack appeared, runnable via pyodide, along with the odd false start (most notably, jyve) at getting a Jupyter server running in the browser. Originally, pyodide had supported its own notebook client (indeed, had been created for it) but that project — iodide — soon languished.

As I haven’t really been Tracking Jupyter since summer last year, there are probably more a few projects ticking along that I missed the earliest of signs of and that have only now come to my attention through occasional mentions on social media that have passed my way.

One of these is jupyterlite (docs), “a JupyterLab distribution that runs entirely in the browser built from the ground-up using JupyterLab components and extensions”. It’s not classic notebook, but it does suggest there’s a running jupyter server available as a WASM component…

So why is this interesting?

To run a Jupyter notebook requires three things:

  • a client in the browser;
  • a Jupyter server to serve the client and connect it to a kernel process;
  • a computing environment to execute code in code cells (the kernel process).

If you access a hosted Jupyter environment, someone else manages the Jupyter server and computing environment for you. If you run notebooks locally, you need at least a Jupyter server, and then you can either connect to a remote kernel or run one locally.

To run a multi-user hosted server, you need to run the Jupyter server, and potentially also manage authentication, persistent storage for users to save their notebooks, and the compute backend to serve the kernel processes. This means you need infrastructure of the hard kind (servers, storage, bandwidth), and you become a provider of insfrastructure of the soft kind (jupyer notebooks as a service).

With Jupyter running in the browser, using something like Jupyterlite, all you need is a web server. Which you’re probably already running. The notebook server now runs in the browser; the kernel now runs in the browser; and the client (JupyterLab) continues to run in the browser, just as it ever did.

In JupyterLite, storage is provided by the local browser storgae, which means you need to work with a single browser. (With many browsers, such as Chrome, now offering browser synchronisation, I wonder if the local storage is synched too? If so, then you can work from any browser you can “log in” to to enable synchronisation services.)

To my mind, this is a huge win. You don’t need to host any compute or storage services to make interactive computing available to your users/students/learners: you just need a webserver. And you don’t even need to run your own: the jupyterlite demo runs using Github pages.

jupyterlite demo running on Github Pages

For open education, this means you can make a computing environment available, in the browser, using just a webserver, without the overhead, or security concerns, of running a compute backend capabale of running arbitrary, user submitted code.

So that’s one thing.

For learners running things locally, they just need a simple web server (I think this requires serving: clicking on an HTML document to open it in a browser may hit browser issues that expect content served with a particular MIME-type).

A simple web server is the sort of thing that can be easily packaged and distributed, but it still presents something of an overhead in terms os downloading, installing and then running the service.

Perhaps simpler would be distributing the application as a cross-platform electron app? As far as I know, jupyerlite isn’t (yet?!) packaged that way, but there is at least one demo out there of pyodide bundled inside an electron app: inureyes/pyodide-console (presentation). So it’s not hard to imagine bundling and distributing jupyterlite the same way, although the practicalities may prove fiddly. (Or they may not…)

“So what?”, you may say. “If we’re giving students access to anaconda anyway, what benefit does this bring?” Leaving aside the huge questions I have about using things like Anaconda, not least their lack of generality compared to distributing environments using docker containers, for example, and notwithstanding the ability to be able to provide computing environments purely withing the browser as noted earlier, the availablity of a Jupyter server and Jupyter kernel running in the browser makes other things possible, or at least, allows us to entertain the idea of other applications with a view to seeing if they are realisable.

Hmm… maybe chrome can provide the webserver itself?

So what might those things be? Off the top of my head, and without any serious thought at all, several things come immediately to mind.

Firstly, the possibilty of the cross-platfrom electron distribution (which is essentially an application container wrapping a chrome browser and a simple web server).

Secondly, and nosing around a little, there are already VS Code extensions that seem to be riffing on using jupyterlite too; so if you have access to VS Code in the browser, you could perhaps also install a pyodide jupyter kernel and run notebooks using that in VS Code in the browser. (I’m not sure if you can host VS Code using just a simple web server or it needs a nodejs server app?)

Thirdly, it’s not hard to imagine a route towards making interactive books avaliable and served just via a web browser. For example, a jupyter book UI where rather than  having to hook up to a remote jupyter server to run (editable) code cells from the page using thebelab you could just run the cells against the WASM run kernel in the browser. (What would be required to make a thebelab like javaascript package that would allow a Jupyter Book to connect to a jupyterlite server running from the same browser tab?) It would then be possible publish a fully interactive textbook using just a simple web server and no other dependencies. The only piece missing from that jigsaw would be a Jupyter Book extension to allow you to save edited code cells into browser storage; and maybe also add in some means of adding / editing additional html cells (then at a later date adding support for markdown, perhaps).

The availability of a thebelab like package to connect to an “in page” Jupyter environment also means we can support on demand executable code from any code bearing HTML page, such as content pages with code examples in a VLE web page, and without the need for backend server support.

Finally, institutionally, jupyterlite makes it possible to publish a simple Jupyter environment directly from the VLE as a “simple” html page, with no compute backend/traditional Jupyter hosting requirement on the backend. The compute/storage requirement must be provided by the end user in the form of a recent browser and a computer that can cope with running the WASM enviornment inside it.

Related: Fragment – Jupyter Book Electron App.

On the WatchList: VisualPython

A fragmentary note to put a watch on Visual Python, a classic Jupyter notebook extension (note that.. a classic notebook extension) to support visual Python programming:

It’s a bit flaky at the moment — the above screenshot shows multiple previews of the selected function code, and the function preview doesn’t properly render things like the function arguments (nor could I get the function to appear in the list of user defined functions), but it’s early days yet.

At first, a blocker to me in terms of suggesting folk internally have at a look at it right now included the apparant inability to define a variable by visual means (all I wanted to do was set a=1) or clear the notebook cells when I wanted to reflow the visual program into the notebook code cell area.

But then I twigged that rather than trying to create a complete program using the visual tools, a better way of using visualpython might be as a helper to code fragments for me in particular use cases.

In the example below, I created the dataframe manually and then used the editor to create a simple plot command that could be inserted into a notebook code cell. The editor picked up on the dataframe I had defined and used that to prepopulate selection lists in the editor.

If the environment becomes the plaything of devs looking to put complex features into the environment, seeing it as a rich power tool for them (and contra to their beliefs, an increasingly hostile environment to novices as more “powerful” features are added and more visual clutter to the environment to scare the hell of users with things that at irrelevant), then the basic usability required for a teaching and learning environment will be lost if users see it as a tool for creating complete programs visually.

For the developers, it’s all too easy to see how the environment could become as much a toy for adding yet more support for yet more packages that can be demonstrated in the environment but never used (because the power users actually prefer using autocomplete in a “proper IDE”) rather than being simplified for use by novices with very, very, very simple programming demands; (just think of the two, three, four lines of code examples that fill the pages of introductory programming text books).

If folk do want a visual editor for data related programming, wouldn’t they use something like Orange, enso, or the new JupyterLab based orchest?

orchest: JupyterLab visual pipleine programming environment

But if you see the Visual Python editor as a tool at the side that essentially operationalises documentation lookup in a way that helps you create opinionated code fragments, where the opinion is essentially a by prodcut of the code templates that are used to generate code from particular visual UI selections, then I think it could be useful as a support tool for creating code snippets, not as an authoring tool for writing a more complete program or computational analysis.

So what will I be watching for? User uptake (proxied by mentions I see of it), some simple documentation, and perhaps a two minute preview video tour (I’m not willing to spend my time on this right now because I think it needs a bit more time in the oven…). The usability should improve as novices get confused and raise issues with how to perform the most basic of tasks and as the noosphere finds a way to conceptualise the sort of usage patterns and workflows that VisualPython supports best.

My intial reaction was a bit negative — it’s too visually complex already for novices, and some really basic usability issues and operations are either missing or broken, if you see it as an editor for creating complete programs. But if you view it as a code generating documentation support tool that lets you hack together a particular code fragment with visual cues that you might otherwise pick up from from documentation, documentation code examples or simple tutorials, then I think it could be useful.

Hmmm… Another thing to try to get my head round in the context of generative workflow tools…