Category: Anything you want

This is Why We Need Info/Digital Literacy…

A couple of weeks ago, an interesting enough weekend piece in the Observer on how Google search results may not always give you the “factual” sort of result you might expect: Google is not ‘just’ a platform. It frames, shapes and distorts how we see the world.

This weekend just gone, an absolute piece of tosh: How to bump Holocaust deniers off Google’s top spot? Pay Google.

Let me summarise for you: “How to bump legitimate news headlines from the front page – pay the Observer”


Doing it Local… Maybe Next Year…

New Year coming up, so time to start mulling over a resolution or two that might actually make a difference*. One of the things I meant to do this year – but didn’t get round to – was working on, which I’d planned to start populating as a demonstrator site for local data led news stories generated from national datasets. My thought was if I could get into some sort of habit around that, I might actually get round to starting to build up a data driven wire service for hyperlocals and local monitoring groups (,, and were all purchased and parked for this…they’re still unpopulated…).

One plan I had for trying to sneak this project up on myself was to pick a data release every day (or at least, one a week on my 0.2FTE not-OU day, which keeps getting leeched away, somehow…) from the UK Gov daily “published statistics” feed and write a Jupyter notebook to start to explore it. Over the course of a year, I should have been able to get through a fair few datasets and start to return to them, and further work up ones I’ve visited before, as well as starting to build up some sort of longitudinal collection. (Here’s one false start on that around NHS datasets. Here’s another placeholder for some notebooks I was going to work up for OnTheWight before we fell out over openness!) Never really happened though..:-( On the other hand, I did start to play with company data again, courtesy of an invite from Global Witness to their “person’s of significant control” datadive, as well as a wondering about Trump, and I’m fired up to start playing with that data again. As the to local data stories and toolkits – maybe next year…

To that end, the presence of several other projects that look set to be ramping up next year may prompt me into action as a form of mild competition and “could I do that?” inspiration. One example is Will Perrin’s Local News Engine, another the Bureau of Investigative Journalism Local Data Lab, to be headed by Times data journalist Megan Lucero. (At the time of writing, it’s not too late to apply for a data journalist or data lab developer role. I’m not sure if they’re also open to speculative applications…? Hmmm….) Both of those projects are funded from the Google Digital News Initiative Innovation Fund, but I’m not sure what, if anything, that means…

My year should also be kickstarted (hopefully) energy level wise with a few days at the reproducible research using Jupyter Notebooks curriculum development hackathon. One of the things I’ll be interested in is the extent to which any curriculum – and resources produced for it – can also be used to support training initiatives around the use of reproducible scripts for national-to-local data wrangling notebooks for use by local journalists, watchdogs, researchers etc. (I suspect the user skill levels the workshop/hackathon will be focussing on are a skill level one or two up from a more amateur (and I use the word advisedly…) audience, but it’ll be interesting to see how accessible we can make things…)

This might also provide an opportunity for me to think about more about using “databoxes”, Raspberry Pi SD card images blown with all you need to get up and running immediately with a particular dataset. Think RPi runnable Infinite Intern SD cards

Also lined up (nearly… fingers crossed) is taking a more detailed look at Parliamentary open data, and how that can be used to support wider research and “holding to account”, as well as policy development. Whilst that will probably involve some amount of poking around in the data, seeing what’s there, and what can be done with it, it might also set the scene for rethinking how consultations and Parliamentary research briefings might work as informal learning resources requiring a critical read…

Hmmm… thinks again… there’s not a lot of 0.8 interest in there, is there…?

A change for me this year was starting to follow a band again, after 20 years off – though that wasn’t one of the resolutions last time round… Maybe finding some ways to start getting involved with promoting again should be on the list for next year…

Trump’s UK Company Holdings – And Concerns About Companies House Director Name Authority Files

A couple of days ago I had the briefest of looks at Companies House data to see what the extent of Trump’s declared (current) corporate roles are in the UK. Not many, it seems. Of the companies with which Trump has a declared officer interest, the list of co-directed companies in his UK empire seems small:


In my code as it currently stands, two directors are the same if they have the same director number according to Companies House records (I think! Need to check… can’t remember if I also added a fuzzy match…).

Unfortunately, Companies House has issues with name authority files (they need to talk to the librarians who’ve been grappling with the question of whether two people with the same, or almost the same, name are actually the same person for ages… “VIAF” is a good keyword to start on…). For example, I strongly suspect these are the same person, given that I found them by mining co-directed companies seeded on two separate Trump companies:

(u’qcmgW-bhHd3TT1MSNuqHIjWBxLI’, 1946, u’TRUMP, Donald J’)
(u’8WlV7G8p1ojhFks_i4ljYwW5WvI’, 1946, u’TRUMP, Donald John’)
(u’65Cc7HAVpXHqcLR_-CczJ80C724′, 1946, u’TRUMP, Donald’)

Or how about:

(u’sj7c-OeX84Ww_JJudaY_D-DZDm4′, 1981, u’TRUMP, Ivanka’)
(u’omdexC3tGVn8JnozQ9ZazJL_MT8′, 1981, u’TRUMP, Ivanka’)
(u’PCrNv-j3ABqrisHsT_PKL3yAlc0′, 1981, u’TRUMP, Ivanka’)

FWIW, Companies House seem to be increasingly of the opinion that month and year discriminators on birthday are plenty, and day doesn’t need to be publicly shared any more (if, indeed, it will still be collected). Occasional name/month/year collisions aside, this may be true (if you’re happy to accept the collisions). But until they sort their authority files out, and use a common director ID (reconcilable to a Person of Significant Control identifier from the PSC register) for the same person, they should be providing as much info as possible to help the rest of us reconcile director identifiers from their inconsistent data.

PS I started to doubt myself that Companies House at least attempts to use the same identifier for the same person, but here’s another example that I’m pretty sure refer to the same person… – note the first result associates 35 appointments with the name:


If you click the top link, you’ll see the appointment dates to the various companies are different, so it’s presumably not as if the commonality arises from the appointments all being declared on the same form. I’m not sure how Companies House reconciles directors, actually? Anyone know (let me know via the comments if you do…). For now, I assume it to be something like a (case insensitive?) exact string match on name, birthdate, and maybe correspondence address (or at least, a recognisable part of it)?

The following records, this time from Formula One co-directed companies, presumably relate to the same person (an accountant…):

(u’keWSNSl6V3Zg2FNV7vPy6BBVPVw’, 1968, u’LLOWARCH, Duncan Francis’)
(u’dzIMC8ot_A9rJThNdKQ5yQC-M3Y’, 1968, u’LLOWARCH, Duncan Francis’)
(u’m5FeeEsclwF0s57UkL2NcB6MIBk’, None, u’LLOWARCH, Duncan’)
(u’S9zuBVuv1LXtbR62_r-x9RzJzRE’, None, u’LLOWARCH, Duncan’)
(u’BTfAza-kduWKPnuUYPDd3w2i9fc’, None, u’LLOWARCH, Duncan’)
(u’3e8laCMUijwG6FdTnqGcDqMsXr4′, None, u’LLOWARCH, Duncan’)
(u’1Qgz-VCSMqjZZgyaibcvBAyGKUU’, None, u’LLOWARCH, Duncan’)

The Future is Bright for Shoplifters With Body Dysmorphia

Full of cold and stuck in the biggest rut going – hits the spot exactly – I stumble across a post from last year, Geographical Rights Management, Mesh based Surveillance, Trickle-Down and Over-Reach, one of the increasingly many dystopian, were it not real, posts on this blog describing stuff that no-one cares about.

I mentally link it to Amazon Go, Amazon’s soon to be opened concept shop where you swipe in, take what you want, and just leave, presumably passing security cameras and a security guards, as your phone automatically picks up the automatically generated bill on your way out, and which just makes me feel cold in a different sense.

(I’m one of the neo-Luddites who refuses to use self-scan tills in supermarkets and self-pay pumps at petrol stations (cos Are Robots Threatening Jobs or Are We Taking Them Ourselves Through Self-Service Automation?).)

In a further round of consolidation, I have a quick peek around what other news I may have missed over the last year or so, seeing who else might know I’ve popped into the physical Amazon store: Google, perhaps (see SearchEngineLand on Google Launches “Store Visits” Metric In AdWords, To Help Prove Online-To-Offline Impact or Under The Hood: How Google AdWords Measures Store Visits, for example), or Facebook (Facebook’s new ads will track which stores you visit).

I also happen across a seriously f*****d up piece of shop furniture, the Skinny Mirror, a fitting room fitting that makes you look thinner than you are, so you feel better and buy whatever it is you’re trying on… (well, not you trying it on, obviously, some weirdly distorted f*****d up re-presentation of yourself). I imagine folk will then grab a selfie using something like the updated version of Facetune, an app that lets you photoshop, (verb), a live preview of yourself before you actually take the photo.

And if they walk out of the not Amazon store without paying, they’ll maybe try to explain it away with “I’ve got the app, so I thought I could just go…”.

PS I really need to put a distorted reality tag on a chunk of stuff on the Digital Worlds blog

PPS ish via @kpfssport, I note a recent report from the University of Leicester that suggests that Mobile Scan and Pay Technology could promote supermarket theft. See also a review of a pay-to-read Australian study (Emmeline Taylor, Supermarket self-checkouts and retail theft: The curious case of the SWIPERS): Are supermarket self-checkouts turning shoppers into swipers?.

New Amazon Developer/Devops Tools, Mobile Targeting

I’ve always found Amazon’s AWS tools really fiddly to use – settings all over the place, the all too easy possibility of putting things into the wrong zone and then forgetting about them/having to try to track them down as you get billed for them, etc etc – but that’s partly the way of self-service, I guess.

Anyway, last week, amongst a slew of other announcements (AI services, new hardware platforms that include FPGAs), Amazon announced a range of developer/devops productivity tools that shows they’re now looking at supporting workflows as well as just providing raw services.

Here’s a quick summary of the ones I spotted:

  • AWS Batch: run batch jobs on AWS;
  • AWS CodeBuild: “a managed build service” that will “build[s] in a fresh, isolated, container-based environment”, incorporating:
    • Source Repository – Source code location (AWS CodeCommit repository, GitHub repository, or S3 bucket).
    • Build Environment – Language / runtime environment (Android, Java, Python, Ruby, Go, Node.js, or Docker).
    • IAM Role – Grants CodeBuild permission to access to specific AWS services and resources.
    • Build Spec – Series of build commands, in YAML form.
    • Compute Type – Amount of memory and compute power required (up to 15 GB of memory and 8 vCPUs).
  • Amazon X-Ray: a debug tool that allows you track things across multiple connected Amazon services. Apparently, Amazon X-Ray provides:

    … follow-the-thread tracing by adding an HTTP header (including a unique ID) to requests that do not already have one, and passing the header along to additional tiers of request handlers. The data collected at each point is called a segment, and is stored as a chunk of JSON data. A segment represents a unit of work, and includes request and response timing, along with optional sub-segments that represent smaller work units (down to lines of code, if you supply the proper instrumentation). A statistically meaningful sample of the segments are routed to X-Ray (a daemon process handles this on EC2 instances and inside of containers) where it is assembled into traces (groups of segments that share a common ID). The traces are segments are further processed to create service graphs that visually depict the relationship of services to each other.

  • AWS Shield: a tool that protects your service against DDoS attacks. In waggish mood, @daveyp suggested that many DDoS attacks he’s aware of come from AWS IP addresses. This feels a bit like a twist on an operating system vendor also selling security software to make up for security deficiencies in their base O/S? That said, “AWS Shield Standard is available to all AWS customers at no extra cost” and seems to be applied in basic mode automatically. Security essentials, then?!

Amazon are also starting to offer segmented alert targeting services for your mobile apps with Amazon Pinpoint. The service lets you “define target segments from a variety of different data sources” and more:

You can identify target segments from app user data collected in Pinpoint. You can build custom target segments from user data collected in other AWS services such as Amazon S3 and Amazon Redshift, and import target user segments from third party sources such as Salesforce via S3.

Once you define your segments, Pinpoint lets you send targeted notifications with personalized messages to each user in the campaign based on custom attributes such as game level, favorite team, and news preferences for example. Amazon Pinpoint can send push notifications immediately, at a time you define, or as a recurring campaign. By scheduling campaigns, you can optimize the push notifications to be delivered at a specific time across multiple time zones. For your marketing campaigns Pinpoint supports Rich Notifications to enable you to send images as part of your campaigns. We also support silent or data notifications which allow you to control app behavior and app config on the background.

Once your campaign is running, Amazon Pinpoint provides metrics to track the impact of your campaign, including the number of notifications received, number of times the app was opened as a result of the campaign, time of app open, push notification opt-out rate, and revenue generated from campaigns.

One thing I didn’t spot were any announcements about significant moves into “digital manufacturing” and 3D print-on-demand (something I wondered about some time ago: Amazon “Edge Services” – Digital Manufacturing).

They do seem to be moving into surveilled, auto-checkout, real-world shopping though… Amazon Go.

Would You Describe Your Relationship With Google, Amazon, or Apple as “Intimate” and/or Their Relationship With You as “Controlling” or “Coercive”?

I’ve been thinking about all those terms and conditions that the big web corps use to justify doing what they want with the data they collect about our actions. And also the way that Facebook, particularly, does abusive stuff and then just apologises, says sorry, it won’t happen again…

From the UK Serious Crime Act 2015, c. 9, Part 5, s. 76:

76 Controlling or coercive behaviour in an intimate or family relationship

(1) A person (A) commits an offence if—

   (a) A repeatedly or continuously engages in behaviour towards another person (B) that is controlling or coercive,

   (b) at the time of the behaviour, A and B are personally connected,

   (c) the behaviour has a serious effect on B, and

   (d) A knows or ought to know that the behaviour will have a serious effect on B.

(2) A and B are “personally connected” if—

   (a) A is in an intimate personal relationship with B, or

   (b) A and B live together and—

   (i) they are members of the same family, or

      (ii) they have previously been in an intimate personal relationship with each other.

(3) But A does not commit an offence under this section if at the time of the behaviour in question—

   (a) A has responsibility for B, for the purposes of Part 1 of the Children and Young Persons Act 1933 (see section 17 of that Act), and

   (b) B is under 16.

(4) A’s behaviour has a “serious effect” on B if—

   (a) it causes B to fear, on at least two occasions, that violence will be used against B, or

   (b) it causes B serious alarm or distress which has a substantial adverse effect on B’s usual day-to-day activities.

(5) For the purposes of subsection (1)(d) A “ought to know” that which a reasonable person in possession of the same information would know.

(6) For the purposes of subsection (2)(b)(i) A and B are members of the same family if—

   (a) they are, or have been, married to each other;

   (b) they are, or have been, civil partners of each other;

   (c) they are relatives;

   (d) they have agreed to marry one another (whether or not the agreement has been terminated);

   (e) they have entered into a civil partnership agreement (whether or not the agreement has been terminated);

   (f) they are both parents of the same child;

   (g) they have, or have had, parental responsibility for the same child.

(7) In subsection (6)

  • “civil partnership agreement” has the meaning given by section 73 of the Civil Partnership Act 2004;

  • “child” means a person under the age of 18 years;

  • “parental responsibility” has the same meaning as in the Children Act 1989;

  • “relative” has the meaning given by section 63(1) of the Family Law Act 1996.

(8) In proceedings for an offence under this section it is a defence for A to show that—

    (a) in engaging in the behaviour in question, A believed that he or she was acting in B’s best interests, and

    (b) the behaviour was in all the circumstances reasonable.

(9) A is to be taken to have shown the facts mentioned in subsection (8) if—

    (a) sufficient evidence of the facts is adduced to raise an issue with respect to them, and

    (b) the contrary is not proved beyond reasonable doubt.

(10) The defence in subsection (8) is not available to A in relation to behaviour that causes B to fear that violence will be used against B.

(11) A person guilty of an offence under this section is liable—

   (a) on conviction on indictment, to imprisonment for a term not exceeding five years, or a fine, or both;

   (b) on summary conviction, to imprisonment for a term not exceeding 12 months, or a fine, or both.

To what extent could the sorts of thing that recommendation services do, (recommendation services that model a great deal about us), start to appear coercive? Can the asymmetric (power) relationship we are in with this services be defined as “intimate”?

PS by the by, I’ve started looking at laws again that might be used as the basis of “robot laws” (laws relating to slavery, animal rights, accessibility, limits on behaviour as a result of mental (in)capacity etc) and also started trying to note the laws that companies use to weasel their way out of various corporate responsibilities. Things like the Innocent publication defence in The Business Protection from Misleading Marketing Regulations 2008, for example; how easy is it to look up whether Google or Facebook have availed themselves of this sort of defence, I wonder?

Forget Fake News – Worry About the Chaff…

According to the Encyclopedia Britannica (online edition) there are several sorts of electronic countermeasure used against opponents’ radar:

Electronic countermeasures (electronic warfare)

The purpose of hostile electronic countermeasures (ECM) is to degrade the effectiveness of military radar deliberately. ECM can consist of (1) noise jamming that enters the receiver via the antenna and increases the noise level at the input of the receiver, (2) false target generation, or repeater jamming, by which hostile jammers introduce additional signals into the radar receiver in an attempt to confuse the receiver into thinking that they are real target echoes, (3) chaff, which is an artificial cloud consisting of a large number of tiny metallic reflecting strips that create strong echoes over a large area to mask the presence of real target echoes or to create confusion, and (4) decoys, which are small, inexpensive air vehicles or other objects designed to appear to the radar as if they are real targets. Military radars are also subject to direct attack by conventional weapons or by antiradiation missiles (ARMs) that use radar transmissions to find the target and home in on it. A measure of the effectiveness of military radar is the large sums of money spent on electronic warfare measures, ARMs, and low-cross-section (stealth) aircraft.

These are worth bearing in mind when using Twitter and other social media, as well as keyword driven news search alerts, as your own, personal news radar. In this analogy, the things I want to detect are “true” news stories (whatever that means…); here are some countermeasures you could take to try to prevent high quality news signals, or news signals that inform me about the things you are doing that you don’t want me to know about, or that you need to spin because they paint you in an unfavourable light, getting through to me:

  • noise jamming: pollute my feed with noise that makes me filter out certain forms of traffic (your noise) and, as a side effect, legitimate news; reference me in e.g. tweets and swamp my mentions feed with noise; if I’ve subscribed to one of the accounts you control, feed that stream with random retweets, auto-generated rubbish, etc;
  • false target generation: try to get me to subscribe to an account you control, thinking it’s a legitimate news source;
  • chaff: chaff masks your current “location”, or a story about you; if I make a search or want to follow a particular topic, try to make sure all I can ever find are empty pages that attract those search terms, or your spin on the story;
  • decoys: push out your own news story or, even better, a ridiculous claim that gets widely reshared and that pulls interest away form a legitimate story breaking about you; if I’m only going to read one thing about you today, better it’s the one you put out rather than the one that shows you for what you are…

(If you can think of better examples, please share them in the comments; this was just a quick coffee break post… didn’t really try to think the examples through…)

Remember, folks, this is information war… We should all be reading up on psyops too…