Practical DigiSchol – Refinding Lost Recent Web History

I don’t know to what extent our current courses equip folk for the future (or even the present), but here’s a typical example of something I needed to figure out a solution to a problem (actually, more of a puzzle) today using the resources I had to hand…

Here’s the puzzle:

I couldn’t remember offhand what this referred to other than “body shape” and “amazon” (I’ve had a cold recently and it seems to have affected my mental indexing a bit…!), and a quick skim of my likely pinboard tags didn’t turn anything up…

Hmmm… I recall looking at a related page, so I should be able to search my browser history. An easy way to do this is using Simon Willison’s datasette app to search the SQLite database that contains my Chrome browser history:

pip3 install datasette
datasette ~/Library/Application\ Support/Google/Chrome/Default/History

This fires up a  simple server with a browser based UI:

Body Labs – that was it…

Sigh… So what was there?

You can browse the archived site on the Wayback Machine.

On reflection (as I said: head cold, not focussing/concentrating properly), I could have web searched:

No closer to the video but it sets the context…

That said, searching for bodylabs video brings up some likely candidates for alternatives, such as this one (pixelfondue: Body + Measure + Scan + AI = Bodylabs delicacy):

Anyway, back to the question: what did I need to know in order to be able to do that? And where do folk learn that sort of thing, whether or not they are formally taught it. Indeed, if there is a related thing to be formally taught, what it is, at what level, and in what context?

PS I had actually also kept a note of the company and the video in a scribble pad draft post on the Digital Worlds blog where I collect irreality related stiff:

Edit_Post_‹_Digital_Worlds_

Extending the folium Magic…

A couple of days ago I posted about some IPython magic for embedding interactive maps in Jupyter notebooks.

I had a bit more of a play yesterday, and then into the night, and started to add support in for other things. For example, you can now build up a map using several pieces of magic, for example by assigning a magic generated map to a variable and then passing that map in as the basemap to another piece of magic.

If the output of the previously run code cell is a folium map, the magic will use that map as the starting map by default – there is a switch (-b None), or some alternative magic (%folium_new_map) that forces a fresh map if you want to start from scratch.

The magic now also has some support for trying to guess where to centre a map, as well as logic that tries to guess at what columns you might want to use when generated a choropleth map from some data and a geojson file. This is best exemplified by some helper magic:

The same routines are used to try to guess column names if you omit them when trying to plot a choropleth. For example, here we only pass in a reference to the data file, the geojson file, and the numeric data column to colour the map. The columns used to plot the boundaries are guessed at.

This is likely to be very ropey – I’ve only tested it with a single pair of data/geosjon files, but it might work for limited general cases…

You can play with it here: Binder

Code is…

…something that everyone is supposed to learn, apparently, but I’m not sure why; and you don’t have to if you’re an adult learner, compared to someone in school?

So here are a couple of reasons why I think knowledge of how to use code is useful.

Code is a means for telling computers, rather than computer users, what to do

Writing instructional material for students that require them to use computer application is a real pain. You have to write laborious From the file menu, select…, in the popup wizard, check the… instructions, or use screenshots (which have to look exactly the same as what the learner is likely to see or they can get worried) or screencasts (which have to look exactly the same as what the learner is likely to see or they can get worried).

The problem is, of course, you don’t want the learner to really do any of that. You want them to be apply to apply a series of transformations to some sort of resource, such as a file.

On the other hand, you can write a set of instructions for what you want the computer to do to the resource, and give that to the learner. The learner can use that medium for getting the computer to transform the resource rather more literally…

Code is a tool for building tools

Code lets you build your own tools, or build your own tools on top of your own tools.

One huge class of tools you can build are tools that automate something else, either by doing a repetitive task multiple times over on your behalf, or by abstracting a lengthy set of instructions into a single instruction with some “settings” passed as parameters. And yes, you do know what parameters are: in the oven for thirty minutes at gas mark 5, which is to say, in “psuedo-code”, oven(gas_mark=5, time=30).

Here’s one tool I built recently – some magic that makes it easy to embed maps in Jupyter notebooks.

Code is a tool for extending other tools

If some you have access to the code that describes a tool, you can extend it, tune it, or customise it with your own code.

For example, here’s that  previous magic extended.

Code is something that can be reused in, or by, other bits of code

If you’ve written some code to perform a particular task in order to achieve something in one tool, you don’t need to write that code again if you want to perform the same task in another tool. You can just reuse the code.

Lots of code is “algorithmic boilerplate”, in the sense that it implements a recipe for doing something. Oftentimes, you may want to reuse the “algorithmic boilerplate” that someone else has got working in their application / tool, and reuse it, perhaps with slight remodification (swapping dataframes for datafiles, for example).

Disclaimer

In my student days, many nights were spent between the computer room, and the Student Union playing bridge. In the bridge games, the bidding was ad hoc because no-one could ever remember any of  the bidding systems you’re “supposed to use” properly… Obviously, it wasn’t proper bridge, but it passed the time amicably…

So unlike folk who (think code should be taught properly and don’t think to use code themselves for daily tasks), I have more pragmatic approach and ((think my code only has to work enough to make it worthwhile using it) or (because it’s an applied recreational activity like doing a crossword)).

Embedding folium Maps In Jupyter Notebooks Using IPython Magic

Whilst trying to show how interactive maps can be embedded in a Jupyter notebook, one of the comments I keep backing back is that “It’s too hard” because you have to write two or three lines of code.

So I’ve tried to simplify things by wrapping the two or three lines of code up as IPython magic, which means you can use a one liner.

The code can be found in this Githib repo: psychemedia/ipython_magic_folium.

To install:

pip install git+https://github.com/psychemedia/ipython_magic_folium.git

To load the magic in a Jupyter notebook:

%load_ext folium_magic

Then call as: %folium_map

The magic currently only works as line magic.

See the folium_magic_demo.ipynb notebook for examples, or run using Binder.

Binder

Display Map

  • -l, --latlong: latitude and longitude values, comma separated. If no value is provided a default location will be used;
  • -z, --zoom (default=10): set initial zoom level;

Add markers

  • -m, --marker: add a single marker, passed as a comma separated string with no spaces after commas; eg 52.0250,-0.7084,"My marker"

-M,--markers: add multiple markers from a Python variable; pass in the name of a variable that refers to:
– a single dict, such as markers={'lat':52.0250, 'lng':-0.7084,'popup':'Open University, Walton Hall'}
– a single ordered list, such as markers=[52.0250, -0.7084,'Open University, Walton Hall']
– a list of dicts, such as markers=[{'lat':52.0250, 'lng':-0.7084,'popup':'Open University, Walton Hall'},{'lat':52.0, 'lng':-0.70,'popup':'Open University, Walton Hall'}]
– a list of ordered lists, such as markers=[[52.0250, -0.7084,'Open University, Walton Hall'], [52., -0.7,'Open University, Walton Hall']]

If no -l co-ordinate is set to centre the map, the co-ordinates of the single marker, or the mid-point of the multiple markers, are used instead.

Display `geojson` file

  • -g, --geojson: path to a geoJSON file

If no -l co-ordinate is set to centre the map, the mid-point of the geojson boundary is used instead.

Display a Choropleth Map

A choropoleth map is displayed if enough information is provided to disaplay one.

  • -g/ --geojson: path to a geoJSON file
  • -d, --data: the data source, either in the form of a pandas dataframe, or the path to a csv data file
  • -c, --columns: comma separated (no space after comma) column names from the data source that specify: column to match geojson key,column containing values to display
  • -k, --key: key in geojson file to match areas with data values in data file;
  • optional:
  • -p, --palette: default='PuBuGn'
  • -o, --opacity: default=0.7

For example, load data from a pandas dataframe:

Or load from a data file:

 

This is still a bit fiddly because it requires you to add lat/longs for the base map and/or markers. But this could probably be addressed (ha!) by building in a geocoder, if I can find one that’s reliable and doesn’t require a key.

Scratch Materials – Using Blockly Style Resources in Jupyter Notebooks

One of the practical issues associated with using the Scratch desktop application (or it’s OU fork, OUBuild) for teaching programming is that runs on the desktop (or perhaps a tablet? It’s an Adobe Air app which I think runs on iOS?). This means that the instructional material is likely to be separated from the application, either as print or as screen based instructional material.

OUBuild

If delivered via the same screen as the application, there can be a screen real estate problem when trying to display both the instructional material and the application.

In OU Build, there can also be issues if you want to have two projects open at the same time, for example to compare a provided solution with your own solution, or to look at an earlier project as you create a new one. The solution is to provide two copies of the application, each running its own project.

Creating instructional materials can also be tricky, requiring the capturing of screenshots from the application and then inserting them in the materials, along with the attendant risk when it comes to updating the materials that screenshots as captured in the course materials may drift from the actuality of the views in the application.

So here are a couple of ways that we might be able to integrate Scratch like activities and guidance into instructional materials.

Calysto/Metakernel Jigsaw Extension for Jupyter Notebooks

The Calysto/Metakernel* Jigsaw extension for Jupyter notebooks wraps the Google Blockly package for use in a Jupyter notebook.

Program code is saved as an XML file, which means you can save and embed multiple copies of the editor within the same Jupyter notebook. This means an example programme can be provided in one embed, and the learner can build up the programme themselves in another, all in the same page.

The code cell input (the bit that contains the %jigsaw line) can be hidden using the notebook Hide Input Cell extension so only the widget is displayed.

The use of the editor is a bit tricky – it’s easy to accidentally zoom in and out, and I’m guessing not very accessible, but it’s great as a scratchpad, and perhaps as an instructional material authoring environment?

Live example on Binderhub

For more examples, see the original Jigsaw demo video playlist.

For creating instructional materials, we should be able to embed multiple steps of a programme in separate cells, hiding the code input cell (that is, the %jigsaw line) and then export or print off the notebook view.

LaTeX Scratch Package

The LaTeX Scratch package provides a way of embedding Blockly style blocks in a document through simple LaTeX script.

Using a suitable magic we can easily add scripts to the document (the code itself could be hidden using the notebook Hide Code Cell Input extension.

(Once again, the code cell input (the cell that contains the lines of LaTeX code) can be hidden using the notebook Hide Input Cell extension so only the rendered blocks are displayed.)

We can also create scripts in strings and then render those using line magic.

Live example on Binderhub

One thing that might be quite interesting is a parser that can take the XML generated from the Jigsaw extension and generate LaTeX script from it, as well as generating a Jigsaw XML file from the LaTeX script?

Historical Context

The Scratch rebuild – OU Build – used in the OU’s new level 1 introductory computing course is a cross platform, Adobe Air application. I’d originally argued that if the earlier taken decision to use a blocks style environment was irreversible, the browser based BlockPy (review and code) application might be a more interesting choice: the application was browser based, allowed users to toggle between blocks and Python code views, displayed Python errors messages in a simplified form, and used a data analysis, rather than animation, context, which meant we could also start to develop data handling skills.

BlockPy

One argument levelled against adopting BlockPy was that it looked to be a one man band in terms of support, rather than the established Scratch community. I’m not sure how much we benefit from, or are benefit to, the Scratch community though? If OU Build is a fork,  we may or may not be able to benefit from any future support updates to the Scratch codebase directly. I don’t think we commit back?

If the inability to render animations had also been a blocker, adding an animation canvas as well as the charting canvas would have been a possibility? (My actual preference was that we should do a bigger project and look to turn BlockPy into a Jupyter client.)

Another approach that is perhaps more interesting from a “killing two birds with one stone” perspective is to teach elementary programming and machine learning principles at the same time. For example, using something like Dale Lane’s excellent Scratch driven Machine Learning for Kids resources.

PS the context coda is not intended to upset, besmirch or provoke anyone involved with OUBuild. It’s self-contempt / self-critical, directed at myself for not managing to engage/advocate my position/vision in a more articulate or compelling way.

Fragment – Breaking Enigma Was Only Part of the Story…

Reading Who, me? They warned you about me? on ethics associated with developing new technologies, this quote jumped out at me: [m]y claim is that putting an invention into a public space inevitably makes that invention safer (Mike Loukides, December 7, 2017).

In recent years, the UK Government had several goes at passing bills that referred to the collection of communications data, the who, where, when and how of a communication but not its content .

[N]ot its content.

Many folk are familiar with stories of the World War Two codebreakers, the boffins, Alan Turing among them, who cracked the German enigma code. How they helped win the war by reading the content of enemy communications.

So given it was the content wot won it, we, cast as “enemies”, might conclude that the protecting the content is key. That the communications data is less revealing.

But that’s not totally true. Other important intelligence can be derived from traffic analysis, looking at communications between actors even if you don’t know the content of the messages.

If I know that X sent a message to Y  and Z five minutes before they committed a robbery on several separate connections, I might suspect that X  knew Y and Z, and was implicated in the crime, even if I didn’t know about the content of the messages.

Location data can also be used to draw similar inferences. For example, the Bloomberg article Mobile-Phone Case at U.S. Supreme Court to Test Privacy Protections describes a recent US Supreme Court case reviewing an appeal from a convicted armed robber who was in part convicted on the basis of evidence that data obtained from [his] wireless carriers to show he was within a half-mile to two miles of the location of four  … robberies when they occurred.

So what has this to do with “putting an invention into a public space”? Perhaps if the stories about how military intelligence made and makes use of traffic analysis and location analysis, and not just the content of decrypted messages, the collection of such data may not seem so innocuous…

When invention takes place in public, we (the public) know that it exists. We can become aware of the risks. Mike Loukides, December 7, 2017.

Just sayin’…

OERs in Practice: Repurposing Jupyter Notebooks as Presentations

Over coffee following a maps SIG meeting last week, fellow Jupyter notebooks enthusiast Phil Wheeler wondered about the extent to which tutors / Associate Lecturers might be able to take course materials delivered as notebooks and deliver them in tutorials as slideshows.

The RISE notebook extension allows notebooks to be presented using the reveal.js HTML presentation framework. The slides essentially provide an alternative client to the Jupyter notebook and can be autolaunched using Binder. (I’m not sure if the autostart slideshow can also be configured to Run All cells before it starts?)

To see how this might work, I marked up one of the notebooks in my showntell/maths demo setting some of the cells to appear as slides in a presentation based on the notebook.

I also used the Hide Input Jupyter extension to hide code cell inputs so that the code used to generate an output image or interactive could be hidden from the actual presentation.

Get into the slideshow editor mode from the notebook View menu, select Cell Toolbar and then Slideshow. Reset the notebook disable using View > Cell Toolbar > None.

To run the presentation with code cell outputs pre-rendered, you first need to run all the cells. From the notebook Cell menu select Run All to execute all the cells.  You can now enter the slideshow using the Enter/Exit RISE Slideshow toolbar buttom (it looks like a bar chart). Exit the presentation using the cross in the topleft of the slideshow display.

[Live demo on Binderhub]

PS building on the idea of using mapping notebook cells into a reveal.js tagged display, I wonder if we could do something similar using a scrollytelling framework such as scrollama, scrollstory or idyll?