Category: Anything you want

Figure Descriptions: Accessibility and Equivalent Experience

One of the things that course teams work hard at at the OU is making materials accessible. This isn’t just because as an educational institution there is a legal obligation to do so: it’s built into the institutional DNA.

In the course of a module production meeting yesterday we had a short workshop on a writing figure descriptions – long text descriptions that can provide a student with a screen reader with an equivalent experience of figure included in the course text, often in the form of a narrated description of the salient points in the image. For readers with a sight impairment, the long description may read out by a screen reader to provide an alternative to looking at the figure directly.

There is an art to writing text descriptions that I’m not sure I’ve ever mastered – I guess I should read the guidance produced by the UK Association for Accessible Formats (which I think draw on OU expertise).

There are some rules of thumb that I do try to bear in mind though (please feel free to correct me in the comments if you take issue with any of these): you don’t want to duplicate what’s in the text that refers to the figure, nor the figure caption. Where the sighted reader is expected to read something for themselves from the figure, you don’t want the figure description to describe the answer as well as the figure. Where the exercise is critiquing a figure, or learning how to read it or extract salient points from it in order to critique it (for example, in the case of an art history course), the long description shouldn’t give away the reading, highlight the salient point specifically, or turn into critique. Generally, the figure description shouldn’t add interpretation to the figure – that comes from the reading of the figure (or the figure description). You also need to take care about the extent which the figure description describes the semantics of the figure; for example, identifying a decision symbol in a flow chart as such (a semantic description) compared to describing it as a diamond (which you might want to do when teaching someone how to read a flow chart for the first time.

Sometimes, a figure appears in a document that doesn’t appear to need much of a description at all; for example, an image that appears purely as an illustration, a portrait of a historical figure, for example, whose brief biographical details appear in the main text. In such a case, it could be argued that a figure description is not really required, or if it is, it should be limited to something along the lines of “A portrait of X”. (A quick way in to generating the description for such an image might also be to refer to any search terms used to discover the image by the original author if it was discovered using a search tool…)

But if the purpose of the image is to break up the flow of the text on the printed page, give the reader a visual break in the text and a brief respite from reading, or help set the atmosphere of the reading, then what should an equivalent experience be for the student accessing the materials via a screen reader? For example, in the workshop I wondered whether the figure description should provide a poetic description to evoke the same sentiment that the author who included the image intended to evoke with it? (A similar trick applied in text is to include a quotation at the start of a section, or as an aside, for example.) A claim could be made that this provides information over and above that contained in the image, but if the aim is to provide an equivalent experience then isn’t this legitimate?

Similarly, if an image is used to lighten the presentation of the text on the page by introducing a break in the text, essentially including an area of white space, how might a light break be introduced into the audio description of the text? By changing the text-to-speech voice, perhaps, or its intonation? On the other hand, an interlude might break a sense of flow if the student is engaged with the academic text and doesn’t want the interruption of a aside?

Another example, again taken from the workshop, concerns the use of photographic imagery that may be intended to evoke a memory of a particular news event, perhaps through the use of an iconic image. In this case, the purpose of the imagery may be emotionally evocative, as well as illustrative; rather than providing a very simple, literal, figure description, could we go further in trying to provide an equivalent experience? For example, could we use a sound effect, perhaps overlaid with a recording of a news headline either taken from a contemporary radio news source (perhaps headed with leading audio ident likely to be familiar to the listener to bring to mind a news bulletin) or a written description then recorded by a voice actor especially to evoke a memory of the event?

In other words, to provide an equivalent experience, should we consider treating the figure description (which will be read by a screen reader) as a radio programme style fill where a sound effect, rather than just a text description, may be more appropriate? For a “poetic aside” intended to substitute for a visual break, should we use a prerecorded, human voice audio clip, rather than triggering the screen reader, even if with a different voice to break up the (audio) flow?

Just as an aside, I note that long descriptions are required for our electronic materials, but I’m not sure how they are handled when materials are produced for print? The OU used to record human readers reading the course texts delivered as audio versions of the course texts to students, presumably with the human reader also inserting the figure descriptions at an appropriate point. I wonder, did the person recording the audio version of the text use a different tone of voice for the different sorts of figures to break up the rest of the recorded text? I also wonder if rather than human reader voiced recordings, the OU now delivers electronic copies of documents that must be converted to speech by students’ own text-to-speech applications? In which case, how do the audio versions compare to the human recorded versions in terms of student experience and understanding?

A couple of other things I wondered about related to descriptions of “annotated” diagrams on the one hand, and descriptions of figures for figures that could be “written” (with the figures generated from the written description) on the other.

In the first case, consider the example of a annotation of a piece of python code, such as the following clumsy annotation of a Python function.


In this case, the figure is annotated (not very clearly!) in such a way to help a sighted reader parse the visual structure of a piece of code – there are semantics in the visual structure. So what’s the equivalent experience for an unsighted or visually impaired student using a screen reader? Such a student is likely to experience the code through a screen reader which will have its own idiosyncratic way of reading aloud the code statement. (There are also tools that can be used to annotate python functions to make them clearer, such as For an unsighted reader using a screen reader, an equivalent experience is presumably an audio annotated version of the audio description of the code that the student might reasonably expect their screen reader to create from that piece of code?

When it comes to diagrams that can be generated from a formally written description of them (such as some of the examples I’ve previously described here), where the figure itself can be automatically generated from the formal text description, could we also generate a long text description automatically? A couple of issues arise here relating to our expectations of the sighted reader for whom the figure was originally created (assuming that the materials are originally created with a sighted reader in mind), such as whether we expect them to be able to extract some sort of meaning or insight from the figure, for example.

As an example, consider a figure that represents a statistical chart. The construction of such charts can be written using formulations such as Leland Wilkinson’s Grammer of Graphics, operationalised by Hadley Wickham in the ggplot2 R library, (or the Yhat python clone, ggplot). I started exploring how we could generate a literal reading of a chart constructed using ggplot (or via a comment, in matplotlib) in First Thoughts on Automatically Generating Accessible Text Descriptions of ggplot Charts in R; a more semantic reading would come from generating text about the analysis of the chart, or describing “insight” generated from it, as things like Automated Insights’ Wordsmith try to do (eg as a Tableau plugin).

Something else I picked up on in passing was that work is ongoing in making maths notation expressed in MathJax accessible via a browser using screen readers (this project maybe? MathJax a11y tool). By the by, it’s perhaps worth noting that MathJax is used to render LaTeX expressions from Jupyter markdown cells, as well as output cells of a Jupyter notebook. In addition, symbolic maths expressions described using sympy are rendered using MathJax. I haven’t tested maths expressions in the notebooks with the simple jupyter-a11y extension though (demo; I suspect it’s just the LaTeX that gets read aloud – I haven’t tested it…) It would be interesting to see hear how well maths expressions rendered in Jupyter notebooks are supported by screen reader tools.

Finally, I realise that I am writing from my own biased perspective and I don’t have a good model in my head for how our unsighted students access our materials – which is more fault me. Apologies if any offence caused – please feel free to correct any misunderstandings or bad assumptions on my part via the comments.

PS one thing I looked for last night but could find were any pages containing example HTML pages along with audio recordings of how a user using a screen reader might hear the page read out. I know I should really install some screen reader tools and try them out for myself, but it would take me time to learn them. Seeing examples of variously complex pages – including ones containing maths expressions, figure descriptions, and so on, and how they sound when rendered using a screen a reader as used by an expert user, would be a useful resource I think?

PPS Of course, when it comes to figure captions for illustrative imagery, we could always give the bots a go; for example, I notice this just appeared on the Google Research blog: Show and Tell: image captioning open sourced in TensorFlow.

The Cost of Scaling…

Via @Charlesarthur, a twitter thread from @nickbaum, one time project manager of Google Reader:

I realized this weekend that it’s my fault that @Google shut down Google Reader. /1

I was the PM from 06-07. We launched a major redesign that significantly changed our growth rate… but didn’t take us to “Google scale”. /2

I used to think it was unfair and short-sighted that Google didn’t give us enough resources to execute to our full potential. /3

… but as a founder, I know resources aren’t something you are owed or deserve. They’re something you earn. /4

I should have realized that not reaching ~100m actives was an existential threat, and worked to convince the team to focus 100% on that. /5

As a service, Google Reader allowed users to curate their own long form content stream by subscribing to web feeds (RSS, Atom). When it shut down, I moved my subscriptions over to, where I still read them every day.

If, as the thread above suggests, Google isn’t interested in “free”, “public” services with less than 100m – 100 million – active users, it means that “useful for some”, even if that “some” counts in the tens of millions, just won’t cut it.

Such are the economics of scale, I guess…

100. million. active. users.

Facebook Clearly States Its Anti-Cultural Stance and Demonstrates Its Intrinsic Threat to Journalism

So…. the Guardian reports: Mark Zuckerberg accused of abusing power after Facebook deletes ‘napalm girl’ post.

Photograph: Nick Ut/AP

“While we recognize that this photo is iconic, it’s difficult to create a distinction between allowing a photograph of a nude child in one instance and not others.

“We try to find the right balance between enabling people to express themselves while maintaining a safe and respectful experience for our global community. Our solutions won’t always be perfect, but we will continue to try to improve our policies and the ways in which we apply them.

That’s what happens when you live by algorithms.

For more on this, see the new book: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy by Mathbabe, Cathy O’Neil.

PS Cf algorithmic false positives, false negatives, wtf – not our fault, it’s algorithmics and we aren’t accountable: Facebook loses legal bid to prevent girl suing over naked picture.

Legally Does It…

Included in the several hundred blog feeds I subscribe to are several legal ones. I don’t really understand the law well enough to read it properly, or think through the consequences of how it might be applied, misapplied, or gamed, so I tend to rely on other commentators for the interpretation and then just skim their reviews for choice phrases or ideas.

So here’s a quick round up of several law related issues that crossed my wires over the last few days, some from law blogs, some not…

First up, it seems Wyoming have been working on a “data trespassing” law – In Wyoming it’s now illegal to collect data about pollution – ENROLLED ACT NO. 61, SENATE SIXTY-THIRD LEGISLATURE OF THE STATE OF WYOMING 2015 GENERAL SESSION:

6-3-414. Trespassing to unlawfully collect resource data; unlawful collection of resource data.
(a) A person is guilty of trespassing to unlawfully collect resource data if he:
(i) Enters onto open land for the purpose of collecting resource data; and
(ii) Does not have:
(A) An ownership interest in the real property or, statutory, contractual or other legal authorization to enter or access the land to collect resource data; or
(B) Written or verbal permission of the owner, lessee or agent of the owner to enter or access the land to collect the specified resource data.
(b) A person is guilty of unlawfully collecting resource data if he enters onto private open land and collects resource data without:
(i) An ownership interest in the real property or, statutory, contractual or other legal authorization to enter the private land to collect the specified resource data; or
(ii) Written or verbal permission of the owner, lessee or agent of the owner to enter the land to collect the specified resource data.

(d) As used in this section:
(i) “Collect” means to take a sample of material, acquire, gather, photograph or otherwise preserve information in any form from open land which is submitted or intended to be submitted to any agency of the state or federal government;
(ii) “Open land” means land outside the exterior boundaries of any incorporated city, town, subdivision approved pursuant to W.S. 18-5-308 or development approved
pursuant to W.S. 18-5-403;

(iv) “Resource data” means data relating to land or land use, including but not limited to data regarding agriculture, minerals, geology, history, cultural artifacts, archeology, air, water, soil, conservation, habitat, vegetation or animal species. “Resource data” does not include data:
(A) For surveying to determine property boundaries or the location of survey monuments;
(B) Used by a state or local governmental entity to assess property values;
(C) Collected or intended to be collected by a peace officer while engaged in the lawful performance of his official duties.
(e) No resource data collected in violation of this section is admissible in evidence in any civil, criminal or administrative proceeding, other than a prosecution for violation of this section or a civil action against the violator.
(f) Resource data collected in violation of this section in the possession of any governmental entity as defined by W.S. 1-39-103(a)(i) shall be expunged by the entity from all files and data bases, and it shall not be considered in determining any agency action.

So, it seems as if you are guilty of trespassing on “open land” if you collect air monitoring or pollution data, for example, for the intention of submitting it to a government agency, state or federal, without permission. And if you do collect it, it can’t be admitted in evidence (and if it is, you presumably admit liability for trespass if you collected it and try to submit it as evidence?); and if that data finds its way into a government database, it has to be deleted and can’t be used by the government entity. Note that “collect” also includes photograph. I’m not sure if drones collecting such data result in the drone operator committing the trespass? Would a drone intrude onto such land? What about aerial photography? Or satellite imagery? Or air / dust data collected outside the boundary on a windy day with the wind blowing across the land in question at you?

One of the things that the blog helps me with is not forgetting. This is handy, not only for cheap told-you-so-years-ago moments, but also keeping track of events that seemed notable at the time, which can help when folk later try to rewrite history. A post today on the IALS Information Law and and Policy blog by Hugh Tomlinson QC – “Right to be forgotten” requires anonymisation of online newspaper archive – reports on a Belgian ruling that seems to have implications for news archives (I don’t count Google’s index as an archive). Apparently:

Digital archiving of an article which was originally lawfully published is not exempt from the application of the right to be forgotten. The interferences with freedom of expression justified by the right to be forgotten can include the alteration of an archived text.

The Court of Appeal had correctly decided that the archiving of the article online constituted a new disclosure of a previous conviction which could interfere with his right to be forgotten.

Balancing the right to be forgotten and the right of the newspaper to constitute archives corresponding to historical truth and of the public to consult these, the applicant should benefit from the right to be forgotten. As the Court of Appeal held, the maintenance of the online article, many years after the events it describes, is likely to cause the applicant disproportionate damage compared to the benefits of the strict respect for freedom of expression.

This is the first case that I am aware of in which a Court has ordered that an online archive should be anonymised – as opposed to the less drastic remedy of ordering the newspaper to take steps to ensure that the article was not indexed by search engines. The Belgian courts were not impressed by arguments in favour of keeping the integrity of online archives.

The English courts have yet to engage with the issue as to whether and to what extent “rehabilitated offenders” should be protected from continuing online dissemination of information about their spent convictions. There are powerful arguments – under both data protection and privacy law – that such protection should be provided in appropriate cases. Online news archives do not possess any “absolute immunity” – they are regularly amended in defamation cases – and effective privacy protection may sometimes require their amendment. It remains to be seen how the English courts will deal with these issues.

What do the librarians think about this?

And what happens when the historical record isn’t? I guess historians really won’t be able to trust press reports as first drafts any more?!

Over on the Inforrm blog, Dan Tench writes about the Digital Economy Bill: new offences for the disclosure of information and the risk to journalists:

Part 5 creates a number of new criminal offences (at clauses 33, 34, 42, 50 and 58) imposing criminal liability on those who receive the information and then disclose it to third parties. For the offence to be committed, the information in question must constitute “personal information”, which is information which relates to and identifies a particular “person”, including a body corporate (clause 32(4)). This is a bizarre definition which means that, contrary to ordinary language and the use of the term in other legal contexts, any information about an identified company would be “personal information” – even something as anodyne as information that a particular company has a number of government contracts.

Defining legal entities such as companies as “persons” to whom data protection clauses apply?! Seriously? (Will that also apply to robots, as per Legislating Autonomous Robots?)

Dan goes on:

Even more significantly, these provisions would also impose criminal liability on the third parties who receive the information if they subsequently disseminate it. In both cases, the offences would be committed even if the disclosure of the information by the original public authority (absent the provisions of the Bill) would not itself constitute a criminal offence.

So imagine if an official at the Environment Agency discloses some information to a say, a local authority, to “improve public service delivery” pursuant to the provisions in clause 29. An individual at the local authority considers that this information reveals a serious iniquity relating a corporate entity and passes it on to a journalist on a national newspaper. The newspaper then publishes the information. It would appear that under these provisions the individual at the local authority, the journalist and most probably the newspaper would all be committing criminal offences.

By contrast, if the official at the Environment Agency had equally taken umbrage with the information in question, he or she had revealed it to the journalist and it had been published on those circumstances, it is unlikely that any offence would have been committed.

There seems no logic in that. It is true that it might be a somewhat rare circumstance when these conditions might apply but making criminal disclosures of any information in any situation is surely something which should be done only with the greatest of care, not least because of the consequences to freedom of expression.

Also today, Out-Law report that the UK government tests whether ‘online activity history’ can serve to verify identity:

“We have been looking at projects that consider the use of different sources of activity history when proving an individual is who they say they are,” [said] Livia Ralph, industry engagement lead at the GDS.

Ralph said that if data from social media accounts can be used for digital ID verification purposes then it could increase UK adults’ use of Verify by 9% and by up to 38% in the case of 16-25 year olds.

Under the Verify system, individuals using government online services choose a certified ID assurance provider with which to verify their identity. This involves answering security questions and entering a unique code sent to an individuals’ mobile number, email address or issued in a call to their fixed-line telephone number.

When using government services online thereafter, government bodies are able to rely on the third party verifications of individuals’ identities. The system is still in development but is aimed at streamlining the identity verification process for both government bodies and the public.

The phrase that jumped out at me first? “When using government services online thereafter, government bodies are able to rely on the third party verifications of individuals’ identities”. And then you just have to flip this to realise that every time you log on to a government or public service, which presumably doesn’t have Facebook (or whoever) tracking set on it, the login will provide Facebook (or whoever) with that information. Good oh – everyone helping everyone else track everyone and everything.

And finally – an email went round the OU a few days ago about some new whistleblowing and anti-fraud policies. One reason for whistleblowing is to get information out about nefarious or fraudulent activities that are either being conducted in secret, or where oversight is failing. I note that public bodies are free to set up operating companies to conduct particular bits of their business (FutureLearn in the OU’s case, for example, or companies set up by local councils). I also note that such companies are not necessarily subject to FOI (the Unison Branch guide to local authority trading companies suggests this is the case if they are not solely owned, for example? FutureLearn is solely owned by the OU – so is it FOIable? It seems so…). With many of the FutureLearn papers tabled to OU committees labeled as “confidential” (and as such not viewable by members of the university not on those committees), presumably on grounds of commercial confidentiality, I wonder more generally about the extent to which universities and public bodies may create companies to limit information sharing? Particularly if such companies come to be classed as “persons” about whom “personal” information, sensitive or otherwise, may not be shared.

Legislating Autonomous Robots

Fifteen years or so ago, now, I worked on an OU short course – T184: Robotics and the Meaning Life. The course took a broad view of robotics, from the technical (physical design, typical control systems, relevant AI – artificial intelligence – techniques (and their limitations), through the social and political consequences. The course also included the RobotLab simulator, which could be used to programme a simple 2D robot, or a HEK – a self-purchased Home Experiment Kit in the form of Lego Mindstorms.

The course was delivered as part of the Technology Faculty Relevant Knowledge programme, originally championed by John Naughton. There’s a lot folk don’t know – or understand – about how the technology world works, and the Relevant Knowledge programme helped address that. The courses were for credit, 10 CAT points at level 1, and were fairly priced: 100 hours for a hundred and fifty quid, and 10 CAT points as a bonus.

One of the things I was keen to put in T184 was a section on robot law, which complemented a section on “robot rights”; this reviewed laws that had been applied to slaves, children, animals and the mentally infirm, “sentient creatures”, in other words, whose behaviour or actions might be the responsibility of someone else, and asked whether such laws might be a useful starting point for legislating around the behaviour of intelligent, self-adaptive robots and their owners / creators. The course also drew on science fiction depictions of robots, making the case that while positronic brains were a fiction, the “Three Laws” that they implemented could be seen as useful design principles for robot researchers:

whereas, until such time, if ever, that robots become or are made self-aware, Asimov’s Laws must be regarded as being directed at the designers, producers and operators of robots, since those laws cannot be converted into machine code;

That phrase does not come from the course, but it does appear in a draft report, published in May this year, from the European Parliament Committee on Legal Affairs [2015/2103(INL)]. The report includes “recommendations to the Commission on Civil Law Rules on Robotics” and, for the EU at least, perhaps acts as a starting pistol for a due consideration of what I assume will come to be referred to as “robot law”.

As well as considering robots as things deserving of rights that could be subjugated, I’d also explored the extent to which robots might be treated as “legal entities” in much the way that companies are legal entities, although I’m not sure that ever made it into the course.

whereas, ultimately, robots’ autonomy raises the question of their nature in the light of the existing legal categories – of whether they should be regarded as natural persons, legal persons, animals or objects – or whether a new category should be created, with its own specific features and implications as regards the attribution of rights and duties, including liability for damage;

Again – that’s the EU report from a couple of months ago. So what exactly is it proposing, and what does it cover? Well, the report:

Calls on the Commission to propose a common European definition of smart autonomous robots and their subcategories by taking into consideration the following characteristics of a smart robot:

  • acquires autonomy through sensors and/or by exchanging data with its environment (inter-connectivity) and trades and analyses data
  • is self-learning (optional criterion)
  • has a physical support
  • adapts its behaviours and actions to its environment;

So not software robots, then? (Which raises a question – how might adaptive algorithms be regulated, and treated under law? Or algorithms that are manifest via “human” UIs, such as conversational chatbots?) Or would such things be argued as having “physical support”?

Hmmm… because whilst the report further notes :

… that there are no legal provisions that specifically apply to robotics, but that existing legal regimes and doctrines can be readily applied to robotics while some aspects appear to need specific consideration;

which is fine, it then seems to go off at a tangent as it:

calls on the Commission to come forward with a balanced approach to intellectual property rights when applied to hardware and software standards, and codes that protect innovation and at the same time foster innovation;

I can see the sense in this, though we maybe need to think about IPR of control models arising from the way an adaptive system is trained, compared to the way it was originally programmed to enable it to be trained and acquire it’s own models, particularly where a third party, rather than a manufacturer, does the training, but then the report seems to go off the rails a bit as it:

calls on the Commission to elaborate criteria for an ‘own intellectual creation’ for copyrightable works produced by computers or robots;

That last sentence surely suggests that they’re talking about algorithms rather than robots? Or are they saying that if I write an adaptive computer program that generates a PNG, it’s not copyrightable, but if I program an adaptive robot with a pen on its back and it draws a picture, that is copyrightable? (I can see the IPR issues here may get a bit messy, though presumably contacts and licenses associated with collaborative generative systems already start to address this?)

The report then seems to go off on another tangent, as it:

Points out that the use of personal data as a ‘currency’ with which services can be ‘bought’ raises new issues in need of clarification; stresses that the use of personal data as a ‘currency’ must not lead to a circumvention of the basic principles governing the right to privacy and data protection;

I’m not sure I see how that’s relevant here? There then follows a few sections relating to specific sorts of robot (autonomous cars, medial robots, drones) before addressing employment issues:

Bearing in mind the effects that the development and deployment of robotics and AI might have on employment and, consequently, on the viability of the social security systems of the Member States, consideration should be given to the possible need to introduce corporate reporting requirements on the extent and proportion of the contribution of robotics and AI to the economic results of a company for the purpose of taxation and social security contributions; takes the view that in the light of the possible effects on the labour market of robotics and AI a general basic income should be seriously considered, and invites all Member States to do so;

So…. robots on the workforce means you have to pay a national insurance contribution for what? FTE human jobs replaced? But there’s also a call for a general basic income?!

Then we return to what I thought the report was about – liability:

Considers that robots’ civil liability is a crucial issue which needs to be addressed at EU level so as to ensure the same degree of transparency, consistency and legal certainty throughout the European Union for the benefit of consumers and businesses alike;

Considers that, in principle, once the ultimately responsible parties have been identified, their liability would be proportionate to the actual level of instructions given to the robot and of its autonomy, so that the greater a robot’s learning capability or autonomy is, the lower other parties’ responsibility should be, and the longer a robot’s ‘education’ has lasted, the greater the responsibility of its ‘teacher’ should be; notes, in particular, that skills resulting from ‘education’ given to a robot should be not confused with skills depending strictly on its self-learning abilities when seeking to identify the person to whom the robot’s harmful behaviour is actually due;

The current recommendation appears to be that liability issues be addressed via a compulsory insurance scheme:

Points out that a possible solution to the complexity of allocating responsibility for damage caused by increasingly autonomous robots could be an obligatory insurance scheme, as is already the case, for instance, with cars; notes, nevertheless, that unlike the insurance system for road traffic, where the insurance covers human acts and failures, an insurance system for robotics could be based on the obligation of the producer to take out an insurance for the autonomous robots it produces;

which is fine, and other paragraphs explore that further; but then the report goes off on one again:

creating a specific legal status for robots, so that at least the most sophisticated autonomous robots could be established as having the status of electronic persons with specific rights and obligations, including that of making good any damage they may cause, and applying electronic personality to cases where robots make smart autonomous decisions or otherwise interact with third parties independently;

Which is firmly in the territory I wanted to explore in T184 way back when. For example, is the suggestion that we have some sort of “Intelligent Robot/Algorithm Capacity Act”, akin to the 2005 Mental Capacity Act perhaps?! Or is it more akin to corporate liability which seems to be under-legislated? And here’s where I start to wonder – where do you distinguish between robots as autonomous things that are legislated against, algorithms as autonomous things that are legislated against, sets of interacting algorithms creating complex adaptive systems as autonomous things that are legislated against, complex adaptive systems such as companies that are legislated against, and so on… (I maybe need to read Iain M. Banks’ sci-fi books about The Culture again!)

The report then goes on to suggest a draft Code of Ethical Conduct for Robotics Engineers, a Licence for Designers and a Licence for Users. But not a Licence for Robots themselves. Nor any mention of the extent to which the built environment should be made accessible for mobile robots. (“Robot accessibility” was another thing I was interested in!;-)

Another document that came out recently hails from the DfT’s Centre for Connected and Autonomous Vehicles is a consultation (in the UK) around Advanced driver assistance systems and automated vehicle technologies: supporting their use in the UK [Pathway to Driverless Cars: Proposals to support advanced driver assistance systems and automated vehicle technologies – PDF]. Apparently:

The only immediate change that we have identified primary legislation that is required now is to update our insurance framework. This will give insurers and manufacturers time to consider what insurance products can come to market in time for when this technology arrives.

This reflects the likely short term arrival of “motorway assist systems for travel on high speed roads (i.e. motorways and major trunk roads); and remote control parking”. Platooning trials are also to take place.

For the longer term, the report distinguishes between “conventional driving, assisted driving and fully automated driving”:



The consultation doc is worth reading in full, but here are a couple of points that jumped out at me:

a vehicle owner who is ‘driving’ the highly automated vehicle might have legitimately disengaged from the driving task, with the vehicle having taken over control. If the technology fails and injures the ‘driver’, the current legislation only requires insurance to cover third parties and not the driver. It is up to the policy owner to seek additional insurance to cover any injury they do to themselves as a result of their own actions or negligence. If the AVT fails then the driver, in effect, becomes a victim as their injuries are not as a result of their own actions or negligence. We therefore need to protect the driver as a potential victim.

So you’ll need to insure yourself against the car?

The last line of this amused me:

We have considered whether a different definition of ‘user’ is needed in the Road Traffic Act for automated vehicles for the purposes of insurance obligation. For the first generation of AVT (where the driver is only ‘hands-off’ and ‘eyes-off’ for parts of the journey) we think that the driver falls under the current definition of a ‘user’. Once fully automated vehicles are available – which would drive themselves for the entire journey – it might be more appropriate to put the insurance obligation solely on the registered keeper.

“Registered keeper”. This may well be the current wording relating to vehicle ownership, but it made me think of a wild animal keeper. So harking back to Robot Law, would it be worth looking at the Dangerous Wild Animals Act 1976 or the Dangerous Dogs Act 1991? (Hmm… code sharing libraries, model sharing algorithms – “breeding” new code from old code…!)

We are not currently proposing any significant change in our rules on liability in road traffic accidents to reflect the introduction of automated cars. We still think a fault based approach combined with existing product liability law, rather than a new strict liability regime, is the best approach for our legal system. We think that the existing common law on negligence should largely be able to adapt to this new technology.

So the car won’t be a legal entity in its own right… though I wonder if a class of vehicles running under the same model/operating system would under the EU approach hinted at above?

If you were of suspicious mind, you might think that there could be an ulterior motive for pushing forward various forms of automative automation…

Data will clearly be required to determine whether the driver or the vehicle was responsible for any collision, such as establishing who was in control at the time of the incident. This is likely to come from in-vehicle data recorders. Many vehicles already have data recorders fitted, although the data collected is not accessible without special equipment.

We expect that the out-of-the-loop motorway driving vehicles that are coming to market soon will have an event data recorder fitted. There are inevitably different views as to what data is essential and of course data protection and privacy considerations are important. It seems likely that data recorders would be regulated on an international basis, like most vehicle technologies. We will participate fully in this debate, equipped with views from the UK manufacturing and insurance industries, evidence from the various trials taking place and the first automated technologies that are coming to market.

Presumably, it’s easiest to just make everyone install a box…. (see Geographical Rights Management, Mesh based Surveillance, Trickle-Down and Over-Reach and Participatory Surveillance – Who’s Been Tracking You Today? for examples of how you can lose control of your car and/or data…) That said, boxes can be useful for crash investigations, and may be used in the defense of the vehicle’s actions, or perhaps in its praise: Tesla’s Autopilot May Have Saved A Life.

The following just calls out to be gamed – and also raises questions around updates, over-the-air or via a factory recall…

We do not think an insurer should be able to avoid paying damages to a third party victim where an automated vehicle owner fails to properly maintain and update the AVT or attempts to circumvent the AVT in breach of their insurance policy. Nor do we think that an insurer should be able to avoid paying damages to a third party victim if the vehicle owner or the named drivers on the policy attempt to use the vehicle inappropriately.

The following point starts to impinge on things like computer misuse as well as emerging robot law?

If an accident occurred as a result of an automated vehicle being hacked then we think it should be treated, for insurance purposes, in the same way as an accident caused by a stolen vehicle. This would mean that the insurer of the vehicle would have to compensate a collision victim, which could include the ‘not at fault driver’ for damage caused by hacking but, where the hacker could be traced, the insurer could recover the damages from the hacker.

In respect of the following point, I wonder of the many products we buy at the moment, how many of them integrate statistical computational models (rather than just rely on physics!)? Is the whole “product liability” thing due a review in more general terms?!

Currently the state of the art defence (section 4(1)(e) of the Consumer Protection Act 1987) provides a defence to product liability if, at the time the product was in the manufacturer’s control, the state of scientific and technical knowledge was not such that a manufacturer could have been expected to discover the defect. We could either leave manufacturers’ liability and product liability as it currently is or, instead, extend the insurance obligation to cover these circumstances so that the driver’s insurance would have to cover these claims.

To keep tabs on the roll out of autonomous vehicles in the UK, see the Driverless vehicles: connected and autonomous technologies policy area.

PS via Ray Corrigan, some interesting future law workshops under the banner Geek Law: Gikll 2013, Gikll 2014, Gikll 2015. The 2016 programme (for the London event, Sept 30) is available in an unreadable font here: Gikll 2016 programme.

Community Detection? (And Is Your Phone a Cookie?)

A few months ago, I noticed that the Google geolocation service would return a lat/long location marker when provided with the MAC address of a wifi router (Using Google to Look Up Where You Live via the Physical Location of Your Wifi Router [code]) and in various other posts I’ve commented on how communities of bluetooth users can track each other’s devices (eg Participatory Surveillance – Who’s Been Tracking You Today?).

Which got me wondering… are there any apps out there that let me detect the MAC address of Bluetooth devices in my vicinity, and is there anyone aggregating the data, perhaps as a quid pro quo for making such an app available?

Seems like the answer is yes, and yes…

For example, John Abraham’s Bluetooth 4.0 Scanner [Android] app will let you [scan] for Bluetooth devices… The information is recorded includes: device name, location, RSSI signal strength, MAC address, MAC address vendor lookup.

In a spirit of sharing, the Bluetooth 4.0 Scanner app “supports the project – crowdsourced Bluetooth database. Users are also reporting usage to find their lost Bluetooth devices”.

So when you run the app to check the presence of Bluetooth devices in your own vicinity, you also gift location of those devices – along with their MAC addresses – to a global database – earthping. Good stuff…not.

We’re all familiar (at least in the UK) with surveillance cameras everywhere, and as object recognition and reconciliation tools improves it seems as if tracking targets across multiple camera views will become a thing, as demonstrated by the FX Pal Dynamic Object Tracking System (DOTS) for “office surveillance”.

It’s also increasingly the case that street furniture is appearing that captures the address of our electronic devices as we pass them. For example, in New York, Link NYC “is a first-of-its-kind communications network that will replace over 7,500 pay phones across the five boroughs with new structures called Links. Each Link will provide superfast, free public Wi-Fi, phone calls, device charging and a tablet for Internet browsing, access to city services, maps and directions”. The points will also allow passers-by to ‘view public service announcements and more relevant advertising on two 55” HD displays’ – which is to say they track everything that passes, tries to profile anyone who goes online via the service, and then delivers targeted advertising to exactly the sort of people passing each link.

LinkNYC is completely free because it’s funded through advertising. Its groundbreaking digital OOH advertising network not only provides brands with a rich, context-aware platform to reach New Yorkers and visitors, but will generate more than a half billion dollars in revenue for New York City.

So I wondered just what sorts of digital info we leak as we do walk down the street. Via Tracking people via WiFi (even when not connected), I learn that devices operate in one of two modes – a listening beacon mode, where they essentially listening for access points, but at high battery cost. Or a lower energy ping mode, where they announce themselves (along with MAC address) to anyone who’s listening.

If you want to track passers-by, many of whom will be pinging their credentials to anyone whose listening, you can set up things like wifi routers in monitor mode to listen out for – and log – such pings. Edward Keeble describes how to do it in the post Passive WiFi Tracking

If you’d rather not hack together such a device yourself, you can always buy something off the shelf to log the MAC addresses of passers-by, eg from somebody such as Libelium’s Meshlium Scanner [datasheet – PDF]. So for example:

  • Meshlium Scanner AP – It allows to detect (sic) Smartphones (iPhone, Android) and in general any device which works with WiFi or Bluetooth interfaces. This model can receive and store data from Waspmotes with GPRS, 3G or WiFi, sending via HTTP protocol. The collected data can be send (sic) to the Internet by using the Ethernet.
  • Meshlium Scanner 3G/GPRS-AP – It allows to detect (sic) Smartphones (iPhone, Android) and in general any device which works with WiFi or Bluetooth interfaces. This model can receive and store data from Waspmotes with GPRS, 3G or WiFi, sending via HTTP protocol. The collected data can be send (sic) to the Internet by using the Ethernet, and 3G/GPRS connectivity
  • Meshlium Scanner XBee/LoRa -AP – It allows to detect (sic) Smartphones (iPhone, Android) and in general any device which works with WiFi or Bluetooth interfaces. It can also capture the sensor data which comes from the Wireless Sensor Network (WSN) made with Waspmote sensor devices. The collected data can be send (sic) to the Internet by using the Ethernet and WiFi connectivity.

So have any councils started installing that sort of device I wonder? And if so, on what grounds?

On the ad-tracking/marketing front, I’m also wondering whether there are extensions to cookie matching services that can match MAC addresses to cookies?

PS you know that unique tat you’ve got?! FBI Develops tattoo tracking technology!

PPS capturing data from wifi and bluetooth devices is easy enough, but how about listening out for mobile phone as phones? Seems that’s possible too, though perhaps not off-the-shelf for your everyday consumer…? What you need, apparently, is an IMSI catcher such as the Harris Corp Stingray. Examples of use here and here.

See also: Tin Foil Hats or Baseball Caps? Why Your Face is a Cookie and Your Data is midata and We Are Watching, and You Will be Counted.