Running lOCL

So I think have the bare bones of a lOCL (local Open Computing Lab) thing’n’workflow running…

I’m also changing the name… to VOCL — Virtual Open Computing Lab … which is an example of a VCL, Virtual Computing Lab, that runs VCEs, Virtual Computing Environments. I think…

If you are Windows, Linux, Mac or a 32 bit Raspberry Pi, you should be able to do the following:

Next, we will install a universal browser based management tool, portainer:

  • install portainer:
    • on Mac/Linux/RPi, run: docker run -d -p 80:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer-ce
    • on Windows, the start up screen suggests docker run -d -p 80:8000 -p 9000:9000 --name=portainer --restart=always -v \\.\pipe\docker_engine:\\.\pipe\docker_engine portainer/portainer-ce may be the way to go?

On my to do list is to customise portainer a bit and call it something lOCL.

On first run, portainer will prompt you for an admin password (at least 8 characters).

You’ll then have to connect to a Docker Engine. Let’s use the local one we’re actually running the application with…

When you’re connected, select to use that local Docker Engine:

Once you’re in, grab the feed of lOCL containers: <s>https://raw.githubusercontent.com/ouseful-demos/templates/master/ou-templates.json</s> (I’ll be changing that URL sometime soon…: NOW IN OpenComputingLab/locl-templates Github repo) and use it to feed the portainer templates listing:

From the App Templates, you should now be able to see a feed of examaple containers:

The [desktop only] containers can only be run on desktop (amd64) processors, but the other should run on a desktop computer or on a Raspberry Pi using docker on a 32 bit Rasbperry Pi operating system.

Access the container from the Containers page:

By default, when you launch a container, it is opened onto the domain 0.0.0.0. This can be changed to the actual required domain via the Endpoints configuration page. For example, my Raspberry Pi appears on raspberrypi.local, so if I’m running portainer against that local Docker endpoint, I can configure the path as follows:

>I should be able to generate Docker images for the 64 bit RPi O/S too, but need to get a new SD card… Feel free to chip in to help pay for bits and bobs — SD cards, cables, server hosting, an RPi 8GB and case, etc — or a quick virtual coffee along the way…

The magic that allows containers to be downloaded to Raspberry Pi devices or desktop machines is based on:

  • Docker cross-builds (buildx), which allow you to build containers targeted to different processors;
  • Docker manifest lists that let you create an index of images targeted to different processors and associate them with a single "virtual" image. You can then docker pull X and depending on the hardware you’re running on, the appropriate image will be pulled down.

For more on cross built containers and multiple architecture support, see Multi-Platform Docker Builds. This describes the use of manifest lists which let us pull down architecture appropriate images from the same Docker image name. See also Docker Multi-Architecture Images: Let docker figure the correct image to pull for you.

To cross-build the images, and automate the push to Docker Hub, along with an appropriate manifest list, I used a Github Action workflow using the recipe decribed here: Shipping containers to any platforms: multi-architectures Docker builds.

Here’s a quick summary of the images so far; generally, they either run just on desktop machines (specifically, these are amd64 images, but I think that’s the default for Docker images anyway? At least until folk start buying the new M1 Macs.:

  • Jupyter notebook (oulocl/vce-jupyter): a notebook server based on andresvidal/jupyter-armv7l because it worked on RPi; this image runs on desktop and RPi computers. I guess I can now start iterating on it to make a solid base Jupyter server image. The image also bundles pandas, matplotlib, numpy, scipy and sklearn. These seem to take forever to build using buildx so I built wheels natively on an RPi and added them to the repo so the packages can be installed directly from the wheels. Pyhton wheels are named according to a convention which bakes in things like the Python version and processor architecture that the wheel is compiled for.

  • the OpenRefine container should run absolutely everywhere: it was built using support for a wide range of processor architectures;

  • the TM351 VCE image is the one we shipped to TM351 students in October; desktop machines only at the moment…

  • the TM129 Robotics image is the one we are starting to ship to TM129 students right now; it needs a rebuild because it’s a bit bloated, but I’m wary of doing that with students about to start; hopefully I’ll have a cleaner build for the February start;

  • the TM129 POC image is a test image to try to get the TM129 stuff running on an RPi; it seems to, but the container is full of all sorts of crap as I tried to get it to build the first time. I should now try to build a cleaner image, but I should really refactor the packages that bundle the TM129 software first because they distribute the installation weight and difficulty in the wrong way.

  • the Jupyter Postgres stack is a simple Docker Compose proof of concept that runs a Jupyter server in one container and a PostgreSQL server in a second, linked container. This is perhaps the best way to actually distribute the TM351 environment, rather than the monolithic bundle. At the moment, the Jupyter environment is way short of the TM351 environment in terms of installed Python packages etc., and the Postgres database is unseeded.

  • TM351 also runs a Mongo database, but there are no recent or supported 32 bit Mongo databases any more so that will have to wait till I get a 64 bit O/S running on my RPi. A test demo with an old/legacy 32 bit Mongo image did work okay in a docker-compose portainer stack, and I could talk to it from the Jupyter notebook. It’s a bit of a pain because it means we won’t be able to have the same image running on 32 and 64 bit RPis. And TM351 requires a relatively recent version of Mongo (old versions lack some essentially functionality…).

Insecure, But Anyway… RPi Web Terminal

A simple http web terminal offering ssh access into a Raspberry Pi…

# In RPi terminal
sudo apt install libffi-dev
pip install webssh

# Run with:
wssh  --fbidhttp=False --port=8000

Then in browser, go to raspberrypi.local:8000 and log in to raspberrypi.local host with defult credentials (user pi, password raspberry), or whatever credentials you have set…

Insecure as anything, but a quick way to get ssh terminal access if you don’t have another terminal handy.

Next obvious steps would be to try to run the service in the background and ideally run it as a service. The security should probably also be tightened up.

Note that another alternative is to run a Jupyter server, which will provide terminal access and some simple auth on the front end, though you’d be limited to running with the permissions associated with the notebook user.

[Ah, this looks like it has steps for getting a service defined, as well as creating a local SSL certificate: https://blog.51sec.org/2020/07/python-development-installation-on.html ]

PS see also webmin: https://sbcguides.com/install-webmin-on-raspberry-pi/ or https://thedreamingdad.com/install-webmin-raspberry-pi/ (although this takes up 300MB+ of space… )

That said, for webmin, Chrome will tell you to f***k off and won’t let you in because the SSL certs will be off… Because Google knows best and Google decides what mortals can and can’t see; and as with everyone else who works in ‘pootahs and IT, it’s not in their interest for folk to have an easy way in to accessing compute via anything other than regulated insitutional or monetised services. I f****g hate people who block access to compute more than I f****g hate computers.)

(Re)Discovering Written Down Legends and Tales of the Isle of Wight

One of the things I’d been hoping to do last year was learn a few Island folklore tales for telling at Island Storytellers sessions. The Thing put paid to those events, of course, but as a sort of new year resolultion, I’ve started digging.

There are a few well worn island tales that appear in pretty much every “tales of the Wight” collection, however it’s themed (smugglers, ghosts, legends, folklore, wrecks, etc) and I guess tales that people still tell within families, so to not just rehash every other story, I figure I need a new way in to some of them.

So I’ve started trying to work up a pattern that takes a place, a time, either a bit of law or a bit of lore, and one or more events as a basis for “researching” a story, from which I can generate:

a) the simple telling, which in many cases may appear on the surface to be a rehash of all the other tellings of the same story;

b) a deeper layer that colours each bit of the story for me and provides more hooks for how to remember it.

Using the place is important because it means I can start to anchor things in a memory palace based on the island. Using the data also provides an opportunity to hook things in the memory palace in temporal layer that allows stories in the same time period to colour and link to each other, as well as stories in the same place to colour the place over time. At some point, I daresay characters may also become pieces in the memory palace.

As far as digging around the stories goes, I’ve started looking for primary and old-secondary resources. Primary in the form of original statutes, places and photos (I intend to visit each location as I pull the pieces together to help situate the story properly), court reports (if I can find them!) etc. And old-secondary sources in the form of old books that tell the now familiar, perhaps even then familiar, tales but from the different historical context of the time of writing.

So for example, there’s a wealth of old tourist guides to the Island, going back a couple of hundred years or so, including the following, which can all be found via the Internet Archive or Google Books:

  • The Isle of Wight: its towns, antiquities, and objects of interest, 1915?
  • Legends and Lays of the Wight, Percy Stone, 1912
  • The Undercliff Of The Isle Of Wight Past And Present, Whitehead, John L. 1911
  • Isle of Wight, Moncrieff, A. R. Hope & Cooper, A. Heaton, 1908
  • Steephill Castle, Ventnor, Isle of Wight, the residence of John Morgan Richards, Esq.; a handbook and a history, Marsh, John, 1907
  • A Driving Tour in the Isle of Wight: With Various Legends and Anecdotes, Hubert Garle, 1905
  • The Isle of Wight, George Clinch, 1904 (2nd edition 1921)
  • The New Forest and the Isle of Wight, Cornish, C. J., 1903
  • A pictorial and descriptive guide to the Isle of Wight, Ward, Lock and Company, ltd, 1900
  • The Isle of Wight, Ralph Darlington 1898
  • Fenwick’s new and original, poetical, historical, & descriptive guide to the Isle of Wight, George Fenwick, 1885
  • A visit to the Isle of Wight by two wights, Bridge, John, 1884
  • Jenkinson’s practical guide to the Isle of Wight, Henry Irwin Jenkinson, 1876
  • Briddon’s Illustrated Handbook to the Isle of Wight, G. Harvey Betts, 1875
  • Nelson’s Handbook to the Isle of Wight: Its History, Topography, and Antiquities, William Henry Davenport Adams, 1873
  • The tourist’s picturesque guide to the Isle of Wight, George Shaw, 1873
  • Mason’s new handy guide to the Isle of Wight, James Mason, 1872
  • Black’s Picturesque Guide to the Isle of Wight, 1871
  • The Isle of Wight, James Redding Ware, 1871
  • Methodism in the Isle of Wight: its origin and progress down to the present times, Dyson, John B 1865
  • The Isle of Wight, a guide, Edmund Venables Rock, 1860
  • The pleasure visitor’s companion in making the tour of the Isle of Wight, pointing out the best plan for seeing in the shortest time every remarkable object, Brannon, 1857
  • Barber’s picturesque guide to the Isle of Wight, Thomas Barber, 1850
  • Bonchurch, Shanklin & the Undercliff, and their vicinities, Cooke, William B., 1849
  • Glimpses of nature, and objects of interest described during a visit to the Isle of Wight, Loudon, Jane, 1848
  • Owen Gladdon’s wanderings in the Isle of Wight, Old Humphrey, 1846
  • A topographical and historical guide to the Isle of Wight, W.C.F.G. Sheridan, 1840
  • Vectis scenery : being a series of original and select views, exhibiting picturesque beauties of the Isle of Wight, with ample descriptive and explanatory letter-press, Brannon, George, 1840
  • The Isle of Wight: its past and present condition, and future prospects, Robert Mudie 1840
  • The Isle of Wight Tourist, and Companion at Cowes, Philo Vectis, 1830
  • Tales and Legends of the Isle Of Wight, Abraham Elder, 1839
  • The beauties of the Isle of Wight, 1826
  • A historical and picturesque guide to the Isle of Wight, John Bullar, 1825
  • A companion to the Isle of Wight; comprising the history of the island, and the description of its local scenery, as well as all objects of curiosity, Albin, John, 1823
  • The delineator; or, A description of the Isle of Wight, James Clarke, 1822
  • A journey from London to the Isle of Wight, Pennant, Thomas, 1801
  • A Tour to the Isle of Wight (two volumes), Charles Tomkins, 1796
  • The history of the Isle of Wight; military, ecclesiastical, civil, & natural: to which is added a view of its agriculture, Warner, Richard, 1795
  • Tour of the Isle of Wight (two volumes), Hassell, John 1790
  • The History of the Isle of Wight, Richard Worsley, 1781 ? 1785

Many of the above recount the same old, same old stories; but from a quick skim, there is often a slightly different emphasis or bit of colourful interpretation.

But the tours also include occasional new stories, and illustrations and fragments of primary material or commentary, and/or references to the same (which will hopefully give me new ratholes to chase down:-).

Many of them also appear to have a fondness for anecdotes about the weather, architecture, landscape, and people encountered, so I’m hopeful of finding some new to me stories in there too…

…such as why there was an Act of James I posted in the entrance to Godshill Church “which enacts that every female who unfortunately intrudes on the parish a second illegitimate child shall be liable to imprisonment and
hard labour in Bridewell for six months”…