Everybody Should Code? Well, Educators Working in Online Learning in a “Digital First” Organisation Probably Should?

Pondering Sheila’s thoughts on Why I Don’t Code?, and mulling over some materials we’re prepping as revisions to the database course…

I see the update as an opportunity to try to develop the ed tech a bit, make it a bit more interactive, make it a bit “responsive” to students so they can try stuff out and get meaningful responses back that helps them check what they’ve been doing has worked. (I’ve called such approaches “self-marking” before, eg in the context of robotics, where it’s obvious if your robot has met the behavioural goal specified in some activity (eg go forwards and backwards the same distance five times, or traverse a square).

Some the best activities in this regard, to my mind at least, can be defined as challenges, with a “low floor, high ceiling” as I always remember Mike Reddy saying. Eg how perfect a square, or how quickly can your robot traverse it. And can it reverse it?

Most kids/students can get the robot to do something, but how well can they can it to do something in particular, and how efficiently? There are other metrics too – how elegant does the code look, or how spaghetti like?!

So prepping materials, I keep getting sidetracked by things we can do to try to support the teaching. One bit I’m doing is getting students to link database tables together, so it seems sensible to try to find, or come up with, a tool that lets students make changes to the database and then look at the result in some meaningful way to check their actions have led to the desired outcome.

There are bits and bobs of things I can reuse, but they require:

1) gluing stuff together to make the student use / workflow appropriate. That takes a bit of code (eg to [make some IPython magic]);(https://github.com/innovationOUtside/ipython_magic_eralchemy) );
2) tweaking things that don’t work quite how want them, either in terms of presentation / styling, or functionality – which means being able to read, as well as write, bits of code;
3) keeping track of work – and thinking – in progress, logging issues, sketching out possible fixes, trying to explore the design space, and use space, including looking for alternative – possibly better – ways of achieving the same thing; which means trying to get other people’s code to work, which in turn means coming up with and trying out small examples, often based on reading documentation, looking for examples on Stack Overflow, or picking through tests to find out how to call what function with what sorts of parameter values.

Code stuff; very practical programmingy code-y stuff. That’s partly how I see code being used – on a daily basis – by educators working in a institution like the OU that claims to pride itself on digital innovation.

It’s also stuff that tests my own knowledge of how things are supposed to work in principle – as well as practice. This plays out through trying to make sure that the tech remains relevant to the educational goal, and ideally helps reinforce it, as well as providing a means for helping the students act even more effectively as independent learners working at a distance…

And to my mind, it’s also part of our remit, if we’re supposed to be trying to make the most use of the whole digital thing… And if we can’t do this in a computing department, then where can – or should – we do it?

PS I should probably have said something about this being relevant to, or an example of, technology enhanced learning, but I think that phrase has been lost to folk who want to collect every bit of data they can from students? Ostensibly, this is for “learning analytics”, which is to say looking at ways of “optimising” students, rather than optimising our learning materials (by doing the basic stuff like looking at web stats as I’ve said repeatedly). I suspect there’s also an element of plan B – if we have a shed load of data, we can sell it to some one as part of every other fire sale.

To Do – Textualisation With Tracery and Database Reporting 2.0

More of a just fragment of things to do, rather than a full post on anything I’ve done, because the blog thing is just increasingly hard to find the time or motivation for now; work sucks time and energy and makes everything just whatever…

Anyway – years ago a post on data textualisation. Nothing’s changed, hype around robot journalism has waned and the offerings there are are still pretty much just your plain old report/document generation.

Note a lot more than homebrew stuff like this, still:

One thing I have noticed is more official Excel spreadsheets starting to include dynmic reporting in them. For example, the NHS Digital Hospital Accident and Emergency Activity, 2016-17 data has an output sheet that will dynamicall create a report from the other data sheets based on a used selection.

The report uses formulas to generate output text:

and tables and charts are also generated as part of the report:

A couple of times, Leigh Dodds has linked out to tracery.io in the data2txt / text generation context, but I’ve yet to play with it (tutorial). There’s a python port at aparrish/pytracery which is probably the one I’ll use.

There’s a notebook that looks like it could provide a handy crib to using pytracery with CSV data. I’m not sure how well it copes with turning numbers into words, but it might be interesting to try to weave in support from something like inflect.py if it doesn’t.

I had a play, when I should have been in the garden… :-( Fork here with modifiers from inflect.py and a demo of using it to generate sentences from rows in a pandas dataframe.

One of the things I’ve started trying to do is package up simple tools to grab structured data from webpublished CSVs and Excel docs in simple SQLite3 databases that can be published using datasette (ouseful-datasupply).

Also on the to do list in this regard is to look at some simple demos for creating datasette templates (perhaps even cribbed from report generating formulas in spreadsheets as a shortcut) that render individual rows (or joined rows) as text reports, such as paragraphs of text reporting on winter sitrep stats for a specified hospital. From one template, we’d auto generate reports for every hospital: database reporting, literally.

Or maybe database reporting, 2.0

(Hmm… datasette reporting…? The practice of generating templated news wire reports using datasette?)

Why 2.0? Because writing database reports has been going on for ever, but getting folk to think about it as a basic journalistic reporting skill that represents a practical example of the sort of thing of everyday task that might become more widespread as a result of “everyone should learn to programme” initiatives.

PS code example using my inflect.py enhanced version of pytracery:

import pandas as pd

df=pd.DataFrame({'name':['Jo','Sam'], 'pos':[1,2]})
df

	name	pos
0	Jo	1
1	Sam	2


rules = {'origin':"#name# was placed #posord.number_to_words#.",
         'posord':'#pos.ordinal#'}

def row_mapper(row, rules):
    row=row.to_dict()
    rules=rules.copy()
    for k in row:
        rules[k] = str(row[k])

    grammar = tracery.Grammar(rules)
    grammar.add_modifiers(base_english)
    return grammar.flatten("#origin#")
    
df['report']=df.apply(lambda row: row_mapper(row, rules), axis=1)
df


	name	pos	report
0	Jo	1	Jo was placed first.
1	Sam	2	Sam was placed second

PPS another example of local data report generation in the wild: Local area SEND report .

Tilted Axes and a Reminder About “Glanceability”

Via the Flowing Data blog (Roger Federer career in rankings and wins) this pretty chart from an SRF story, 20 Jahre, 20 Titel.

The rotated axes balance the chart and make the achievements of Connors, Federer and Lendl more “central” to the story. (SRF are a Swiss news organisation…)

I quickly copied and rotated the chart, and here’s how it look with the axes arranged more traditionally:

The composition feels unbalanced, and at a glance feels like there is less of a story in the chart. (Glanceability… that takes me back…. I first heard the phrase from James Cridland – Learnin’ from Virgin – then, as I noted in Powerpoint Presentations That Support Glanceability, Lorcan Dempsey dug into it a little bit more: Glanceability.)

It also reminds me of “banking to 45 degrees”, “the idea that the average line slope in a line chart should be 45º. This has been dubbed banking to 45º and has turned into one of the bits of common wisdom in visualization as determining the ideal aspect ratio [although it more specifically relates to] the comparison between the slopes of two lines, and the slope is the average between those two lines. So if the goal is to be able to compare the rates of change between lines, the 45º average slope makes sense as a rule” Robert Kosara blog post on Aspect Ratio and Banking to 45 Degrees.

The original statement comes from Cleveland, W.S., McGill, M.E. and McGill, R., (1988) The shape parameter of a two-variable graph, Journal of the American Statistical Association, 83(402), pp.289-300 [JSTOR],  and I think I should probably read it again…

By the by, SRF are one the best news orgs I know for sharing their working – the recipes for the above story can be found on their Gihub repo at srfdata/2018-01-roger-federer – so I could probably have recreated the unrotated chart directly from that source, if I’d had the time to play.

PS see also: Five ways to read a scatterplot on the Datawrapper blog.

Decompressing zipped Javascript

Because it took me soooo long to figure this out…

Zipped Javascript pulled from a URL using python requests; requests.get assigned to r.

#via https://stackoverflow.com/a/28642346/454773

import gzip
import base64

jsonTxt = gzip.decompress(base64.b64decode(r.text))

#using zlib.compress gives an error: lib.error:
#Error -3 while decompressing data: incorrect header check

Does Transfer Learning with Pretrained Models Lead to a Transferable Attack?

Reading a post just now on Logo detection using Apache MXNet, a handy tutorial on how to train an image classifier to detect brand logos using Apache MXNet, a deeplearning package for Python, I noted a reference to the MXNet Model Zoo.

The Model Zoo is an ongoing project to collect complete models [from the literature], with python scripts, pre-trained weights as well as instructions on how to build and fine tune these models. The logo detection tutorial shows how training your own network with a small number of training images is a bit rubbish, but you can make the most of transfer learning to take a prebuilt model that has been well trained and “top it up” with your own training samples. The guess the main idea is: the lower layers of the original model will be well trained to recognise primitive image features, and can be reused, and the final model tweaked to reweight these lower level features in the upper layers so the overall model works with your particular dataset.

So given the ability to generate adversarial examples that trick a model into seeing something that’s not there,  how susceptible will models built using transfer learning on top of pretrained models be to well honed attacks developed on that pretrained model? To what extent will the attacks work out of the can (and/or to what extent) or how easily will they be transferred?

To read:

 

Cognitive Science, 2.0? Google Psychlab

Whilst I was rooting around looking for things to do the year or two after graduating, I came across the emerging, inter-disciplinary field of cognitive science, which used ideas from cognitive psychology, philosophy, AI, linguistics and neuroscience to try to understand “mind and consciousness stuff”. I read Dennett and Searle, and even toyed with going to do a Masters at Indiana, where David Chalmers had been doing all manner of interesting things as a PhD student.

I was reminded of this yesterday whilst reading a post on the Google DeepMind blog – Open-sourcing Psychlab – which opened in a style that began to wind me up immediately:

Consider the simple task of going shopping for your groceries. If you fail to pick-up an item that is on your list, what does it tell us about the functioning of your brain? …

What appears to be a single task actually depends on multiple cognitive abilities. We face a similar problem in AI research …

To address this problem in humans, psychologists have spent the last 150 years designing rigorously controlled experiments aimed at isolating one specific cognitive faculty at a time. For example, they might analyse the supermarket scenario using two separate tests – a “visual search” test that requires the subject to locate a specific shape in a pattern could be used to probe attention, while they might ask a person to recall items from a studied list to test their memory. …

“To address this problem humans”… “rigorously controlled”, pah! So here we go: are Google folk gonna disrupt cognitive psychology by turning away from the science and just throwing a bunch of numbers they’ve managed to collect from wherever, howsoever, into a couple of mathematical functions that tries to clump them together without any idea about what any clusters or grouping mean, or what they’re really clustering around…?

We believe it is possible to use similar experimental methods to better understand the behaviours of artificial agents. That is why we developed Psychlab [ code ], a platform built on top of DeepMind Lab, which allows us to directly apply methods from fields like cognitive psychology to study behaviours of artificial agents in a controlled environment. …

Psychlab recreates the set-up typically used in human psychology experiments inside the virtual DeepMind Lab environment. This usually consists of a participant sitting in front of a computer monitor using a mouse to respond to the onscreen task. Similarly, our environment allows a virtual subject to perform tasks on a virtual computer monitor, using the direction of its gaze to respond. This allows humans and artificial agents to both take the same tests, minimising experimental differences. It also makes it easier to connect with the existing literature in cognitive psychology and draw insights from it.

So, to speed up the way Google figures out how to manipulate folks’ human attention through a screen, they’re gonna start building cognitive agents that use screens as an interface (at first), develop the models so they resemble human users (I would say: “white, male, 20s-30s, on the spectrum”, but it would be more insidious perhaps to pick demographics relating to “minority” groups that power brokers (and marketers) would more readily like to “influence” or “persuade”. But that would be a category mistake, because I don’t think cognitive psychology works like that.), then start to game the hell out of them to see how you can best manipulate their behaviour.

Along with the open-source release of Psychlab we have built a series of classic experimental tasks to run on the virtual computer monitor, and it has a flexible and easy-to-learn API, enabling others to build their own tasks.

Isn’t that nice of Google. Tools to help cog psych undergrads replicate classic cognitive psychology experiments with their computer models.

Each of these tasks have been validated to show that our human results mirror standard results in the cognitive psychology literature.

Good. So Google has an environment that allows you to replicate experiments from the literature.

Just remember that Google’s business is predicated on developing ad tech, and ad revenue in turn is predicated on find ways of persuading people to either persist in, or modify, their behaviour.

And once you’ve built the model, then you can start to manipulate the model.

When we did the same test on a state-of-the-art artificial agent, we found that, while it could perform the task, it did not show the human pattern of reaction time results. .. this data has suggested a difference between parallel and serial attention. Agents appear only to have parallel mechanisms. Identifying this difference between humans and our current artificial agents shows a path toward improving future agent designs.

“Agents appear only to have parallel mechanisms”. Erm? The models that Google built appear only to have parallel mechanisms?

This also makes me think even more we need to rebrand AI. Just like “toxic” neural network research required a rebranding as “deep learning” when a new algorithmic trick and bigger computers meant bigger networks with more impressive results than previously, I think we should move to AI as meaning “alt-intelligence” or “alternative intelligence” in the sense of “alternative facts”.

“[A] path toward improving future agent designs.” That doesn’t make sense? Do they mean models that more closely represent human behaviour in terms of the captured metrics?

What would be interesting would be if the DeepMind folk found they hit a brick wall with deep-learning models and that they couldn’t find a way to replicate to human behaviour. Because that might help encourage the development of “alternative intelligence” critiques.

Psychlab was designed as a tool for bridging between cognitive psychology, neuroscience, and AI. By open-sourcing it, we hope the wider research community will make use of it in their own research and help us shape it going forward.

Hmmm…. Google isn’t interested in understanding how people work from a point of view of pure inquiry. It wants to know how you work so it can control, or at least influence, your behaviour. (See also Charlie Stross, Dude, you broke the future!.)

“The Follower Factory” – Way Back When vs Now

A news story heavily doing the rounds at the moment in the part of the Twitterverse I see is a post from the New York Times called The Follower Factory which describes how certain high profile figures feel the need to inflate their social media follower counts by purchasing followers.

Such behaviour isn’t new, but the story is good one – a useful one – to re-tell every so often. And the execution is a nice bit of scrollytelling.

A few years ago, I posted a recipe for generating such charts (Estimated Follower Accession Charts for Twitter) and applied it to the followers of UK MPs at the time, cross-referencing spikes in follower acquisition against news items in the days before, and on the day of, and hike in numbers or the rate of acquisition of numbers (What Happened Then? Using Approximated Twitter Follower Accession to Identify Political Events).

My thinking at the time was that bursts in follower acquisition, or changes in rate of follower acquisition, might correlate with news events around the person being followed. (It turned out that other researchers had followed a similar idea previously: We know who you followed last summer: inferring social link creation times in twitter.)

Whilst it is trivial for Twitter to generate such reports – they own the data – it is harder for independent researchers. When I charted the follower accession curves for UK MPs, I had access to a whitelisted Twitter API count that meant I could download large amounts of data quite quickly. The (generic) rate limiting constraints on my current Twitter account mean that grabbing the data to generate such charts in a reasonable amount of time nowadays would be all but impossible.

There are several ways round this: one is to purchase the data using a service such as Export Tweet; one is to abuse the system and create my own army of (“fake”) Twitter accounts in order to make use of their API limits to mine the Twitter API using a distributed set of accounts under my control; a third is to “borrow” the rate limits of other, legitimate users.

For example, many sites now offer “Twitter analysis” services if you sign in with your Twitter account and grant the service provider permission to use your account to access various API calls through your account. I imagine that one of the ways such services cover the costs of their free offerings is to make use of API calls generated from user accounts to harvest data to build a big database that more valuable services can be provided off the back of.

In this case, whilst the service is free, the aim is not specifically to collect data about the user so they can be sold to advertisers as part of a specific audience or market segment, but instead to make use of the user’s access to API services so that the service provider can co-opt the user’s account to harvest data from Twitter in a distributed way. That is, the service provider gains the necessary benefits from each user to cover the costs of servicing the needs of that user by gaining access to Twitter data more generally, using the the user’s account as a proxy. The mass of data can then be mined and analysed to create market segments, or exploited otherwise.

This approach to exploiting users by means of exploiting their access to a particular resource is also being demonstrated elsewhere. For example, a Wired post from October last year on “cryptojacking” – Your Browser Could Be Mining Cryptocurrency For A Stranger describes how users’ computers can be exploited via their browsers in the form of Javascript code embedded in web pages or adverts that “steals” (i.e. appropriates, or co-opts) CPU cycles (and the electricity used to power them) to do work on behalf of the exploiter – such as mining bitcoin. A more recent story elsewhere – The Coming Battle for Your Web Browser [via @euroinfosec] – describes how the same sort of exploitation can be used to co-opt your browser to take part in distributed denial of service attacks (the provision of which is another thing that can be bought and sold, and that can hence be used to generate revenue for the provider of that service). Defensive countermeasures are available.

PS I notice @simonw has been grabbing his Twitter follower data into a datasette. Thinks: maybe I should tidy up my ESP code for generating follower networks and use SQLite, rather than MongoDB, for that? I think I had an ESP to do list to work on too..?