Archive for the ‘Policy’ Category
Whenever a new open data dataset is released, the #opendata wires hum a little more. More open data is a Good Thing, right? Why? Haven’t we got enough already?
In a blog post a few weeks ago, Alan Levine, aka @cogdog, set about Stalking the Mythical OER Reuse: Seeking Non-Blurry Videos. OERs are open educational resources, openly licensed materials produced by educators and released to the world so others could make use of them. Funding was put into developing and releasing them and then, … what?
OERs. People build them. People house them in repositories. People do journal articles, conference presentations, research on them. I doubt never their existence.
But the ultimate thing they are supposed to support, maybe their raison d’être – the re use by other educators, what do we have to show for that except whispered stories, innuendo, and blurry photos in the forest?
Alan went in search of the OER reuse in his own inimitable way…
… but came back without much success. He then used the rest of the post to put out all for stories about how OERs have actually been used in the world… Not just mythical stories, not coulds and mights: real examples.
So what about opendata – is there much use, or reuse, going on there?
It seems as is more datasets get opened every day, but is there more use every day, first day use of newly released datasets, incremental reuse of the datasets that are already out, linkage between the new datasets and the previously released ones.
Yesterday, I spotted via @owenboswarva the release of a dataset that aggregated and normalised data relating to charitable grant awards: A big day for charity data. Interesting… The supporting website – 360 Giving – (self-admittedly in it’s early days) allows you to search by funder, recipient or key word. You have to search using the right keywords, though, and the right capitalisation of keywords…
And you may have to add in white space.. so *University of Oxford * as well as *University of Oxford*.
I don’t want to knock the site, but I am really interested to know how this data might be used. Really. Genuinely. I am properly interested. How would someone working in the charitable sector use that website to help them do something? What thing? How would it support them? My imagination may be able to go off on crazy flights of fancy in certain areas, but my lack of sector knowledge or a current headful of summer cold leaves me struggling to work out what this website would tangibly help someone to do. (I tried to ask a similar question around charities data before, giving the example of Charities Commission data grabbed from OpenCharities, but drew a blank then.) Like @cogdog in his search for real OER use case stories, I’d love to hear examples of real questions – no matter how trivial – that the 360 Giving site could help answer.
As well as the website, 360 Giving folk provide a data download as a CSV file containing getting on for a quarter of a million records. The date stamp on the file I grabbed is 5th June 2014. Skimming through the data quickly – my own opening conversation with it can be found here: 360 Giving Grant Navigator – Initial Data Conversation – I noticed through comparison with the data on the website some gaps…
- this item doesn’t seem to appear in the CSV download, perhaps because it doesn’t appear to have a funder?
- this item on the website has an address for the recipient organisation, but the CSV document doesn’t have any address fields. In fact, on close inspection, the record relates to a grant by the Northern Rock Foundation, and I see no records from that body in the CSV file?
- Although there is a project title field in the CSV document, no project titles are supplied. Looking through a sample of grants on the website, are any titles provided?
- The website lists the following funders:
Arts Council England
Arts Council Wales
Heritage Lottery Fund
Northern Rock Foundation
Paul Hamlyn Foundation
Sport Northern Ireland
The CSV file has data from these funders:
Arts Council England
Arts Council Wales
Sport Northern Ireland
That is, the CSV contains a subset of the data on the website; data from Heritage Lottery Fund, Indigo Trust, Northern Rock Foundation, Paul Hamlyn Foundation doesn’t seem to have made it into the data download? I also note that data from the Research Councils’ Gateway to Research (aside from the TSB data) doesn’t seem to have made it into either dataset. For anyone researching grants to universities, this could be useful information. (Could?! Why?!;-)
- No company numbers or Charity Numbers are given. Using opendata from Companies House a quick join on recipient names and company names from the Companies House register (without any attempts at normalising out things like LTD and LIMITED – that is, purely looking for an exact match) gives me just over 15,000 matched company names (which means I now have their address, company number, etc. too). And presumably if I try to match on names from the OpenCharities data, I’ll be able to match some charity numbers. Now both these annotations will be far from complete, but they’d be more than we have at the moment. A question to then ask is – is this better or worse? Does the dataset only have value if it is in some way complete? One of the clarion calls for open data initiatives has been to ‘just get the data out there’ so that it can be started to be worked on, and improved on. So presumably having some company numbers of charity numbers matched is a plus?
Now I know there is a risk to this. Funders may want to not release details about the addresses of the charities of they are funding because that data may be used to plot maps to say “this is where the money’s going” when it isn’t. The charity may have a Kensington address and the received funding for an initiative in Oswaldtwistle, but the map might see all the money sinking into Kensington; which would be wrong. But that’s where you have to start educating the data users. Or releasing data fields like “address of charity” and “postcode area of point of use”, or whatever, even if the latter is empty. As it is, if you give me a charity or company name, I can look up it’s address. And its company or charity number if it has one.
As I mentioned, I don’t want to knock the work 360 Giving have done, but I’m keen to understand what it is they have done, what they haven’t done, and what the opendata they have aggregated and re-presented could – practically, tractably, tangibly – be used for. Really used for.
Time to pack my bags and head out into the wood, maybe…
Some rambling but possibly associated thoughts… I suggest you put Alice’s Restaurant on…
For some time now, I’ve had an uncomfortable feeling about the asymmetries that exist in the open data world as well as total confusion about the notion of transparency.
Part of the nub of the problem (for me) lies with the asymmetric disclosure requirements of public and private services. Public bodies have disclosure requirements (eg Local Government Transparency Code), private companies don’t. Public bodies disclose metrics and spend data, data that can be used in public contract tendering processes by private bodies against public ones tendering for the same service. The private body uses this information – and prices in a discount associated with not having to carry the cost of public reporting – into the bid. The next time the contract is tendered, the public body won’t have access to the (previously publicly disclosed) information that the private body originally had when making its bid. Possibly. I don’t know how tendering works. But from the outside, that’s how it appears to me. (Maybe there needs to be more transparency about the process?)
Open data is possibly a Big Thing. Who knows? Maybe it isn’t. Certainly the big consulting firms are calling it as something worth squillionty billionty of pounds. I’m not sure how they cost it. Maybe I need to dig through the references and footnotes in their reports (Cap Gemini’s Open Data Economy: Unlocking Economic Value by Opening Government and Public Data, Deloitte’s Open growth: Stimulating demand for open data in the UK or McKinsey’s Open data: Unlocking innovation and performance with liquid information). I don’t know how much those companies have received in fees for producing those reports, or how much they have received in consultancy fees associated with public open data initiatives – somehow, that spend data doesn’t seem to have been curated in a convenient way, or as a #opendatadata bundle? – but I have to assume they’re not doing it to fleece the public bodies and tee up benefits for their other private corporate clients.
Reminds me – I need to read Owen Boswarva’s Who supports the privatisation of Land Registry? and ODUG benefits case for open data release of an authoritative GP dataset again… And remind myself of who sits on the Open Data User Group (ODUG), and other UK gov departmental transparency boards…
And read the FTC’s report Data Brokers: A Call For Transparency and Accountability…
Just by the by, one thing I’ve noticed about a lot of opendata releases is that, along with many other sorts of data, they are most useful when aggregated over time or space, and/or combined with other data sets. Looking at the month on month reports of local spending data from my local council is all very well, but it gets more interesting when viewed over several months or years. Looking at the month on month reports of local spending data from my local council is all very well, but it gets more interesting when looking at spend across councils, as for example in the case of looking at spend to particular companies.
Aggregating public data is one of the business models that helps create some of the GDP figure that contributes to the claimed, anticipated squillionty billionty pounds of financial benefit that will arise from open data – companies like opencorporates aggregating company data, or Spend Network aggregating UK public spending data who hope to start making money selling products off the back of public open data they have curated. Yes – I know a lot of work goes in to cleaning and normalising that data, and that exploiting the data collection as a whole is what their business models are about – and why they don’t offer downloads of their complete datasets, though maybe licenses require they do make links to, or downloads of, the original (“partial”) datasets available?
But you know where I think the real value of those companies lies? In being bought out. By Experian, or Acxiom (if there’s even a hint of personally identifiable data through reverse engineering in the mix), or whoever… A weak, cheap, cop out business model. Just like this: Farmers up in arms over potential misuse of data. (In case you missed it, Climate Corporation was one of the OpenData500 that aggregated shed loads of open data – according to Andrew Stott’s Open Data for Economic Growth report for the World Bank, Climate Corp “uses 60 years of detailed crop yield data, weather observations from one million locations in the United States and 14 terabytes of soil quality data – all free from the US Government – to provide applications that help farmers improve their profits by making better informed operating and financing decisions”. It was also recently acquired by Monsanto – Monsanto – for just under a billion US $. That’s part of the squillionty billionties I guess. Good ol’ open data. Monsanto.)
Sort of related to this – that is, companies buying others to asset strip them for their data – you know all that data of yours locked up in Facebook and Google? Remember MySpace? Remember NNTP? According to the Sophos blog, Just Because You Don’t Give Your Personal Data to Google Doesn’t Mean They Can’t Acquire It. Or that someone else might buy it.
And as another aside – Google – remember Google? They don’t really “read” your email, at least, people don’t, they just let algorithms process it so the algorithms can privately just use that data to send you ads, but no-one will ever know what the content of the email was to trigger you getting that ad (‘cos the cookie tracking, cookie matching services can’t unpick ad bids, ad displays, click thrus, surely, can they?!), well – maybe there are side effects: Google tips off cops after spotting child abuse images in email (for some reason, after initially being able to read that article, my browser can’t load it atm. Server fatigue?). Of course, if Google reads your ads for blind business purposes and ad serving is part of that blind process you accept it. But how does the law enforcement ‘because we can even though you didn’t warrant us to?’ angle work? Does the Post Office look inside the envelope? Is surveillance actually part of Google’s business model?
If you want to up the paranoia stakes, this (from Ray Corrigan, in particular: “Without going through the process of matching each government assurance with contradictory evidence, something I suspect would be of little interest, I would like to draw your attention to one important misunderstanding. It seems increasingly to be the belief amongst MPs that blanket data collection and retention is acceptable in law and that the only concern should be the subsequent access to that data. Assertions to this effect are simply wrong.”) + that. Because one day, one day, they may just find your name on an envelope of some sort under a tonne of garbage. Or an algorithm might… Kid.
But that’s not what this post is about – what this post is about is… Way back when, so very long ago, not so very long ago, there was a license called GPL. GPL. And GPL was a tainting license. findlaw describes the consequences of reusing GPL licensed code as follows: Kid, ‘if a user of GPL code decides to distribute or publish a work that “in whole or in part contains or is derived from the [open source] or any part thereof,” it must make the source code available and license the work as a whole at no charge to third parties under the terms of the GPL (thereby allowing further modification and redistribution).
‘In other words, this can be a trap for the unwary: a company can unwittingly lose valuable rights to its proprietary code.’
Now, friends, GPL scared people so much that another license called LGPL was created, and LGPL allowed you to use LGPL licensed code without fear of tainting your own code with the requirement to open up your own code as GPL would require of it. ‘Cos licenses can be used against you.
And when it comes to open data licenses, they seem to be like LGPL. You can take open public data and aggregate it, and combine it, and mix it and mash it and do what you like with it and that’s fine… And then someone can come along buy that good work you’ve done and do what they want with it. Even Monsanto. Even Experian. And that’s good and right, right? Wrong. The ODUG. Remember the ODUG? The ODUG is the Open Data User Group that lobbies government for what datasets to open up next. And who’s on the ODUG? Who’s there, sitting there, on the ODUG bench, right there, right next to you?
Kid… you wanna be the all-open, all-liberal open data advocate? You wanna see open data used for innovation and exploitation and transparency and all the Good Things (big G, big T) that open data might be used for? Or you wanna sit down on the ODUG bench? With Deloitte, and Experian, and so on…
And if you think that using a tainting open data license so anyone who uses that data has to share it likewise, aggregated, congregated, conjugated, disaggregated, mixed, matched, joined, summarised or just otherwise and anyways processed, is a Good Thing…? Then kid… they’ll all move away from you on the bench there…
Because when they come to buy you, they won’t your data to be tainted in any way that means they’ll have to give up the commercial advantage they’ll have from buying up your work on that open data…
But this post? That’s not what this post is about. This post about holding companies to account. Open data used to hold companies to account. There’s a story to be told that’s not been told about Dr Foster, and open NHS data and fear-mongering and the privatisation of the NHS and that’s one thing…
But another thing is how government might use data to help us protect ourselves. Because government can’t protect us. Government can’t make companies pay taxes and behave responsibly and not rip off consumers. Government needs our help to do that. But can government help us do that too? Protect and Survive.
There’s a thing that DECC – the Department of Energy and Climate Change – do, and that’s publish statistics about domestic energy price statistics and industrial energy price statistics and road fuel and other petroleum product price statistics, and they’re all meaningless. Because they bear little resemblance to spot prices paid when consumers pay their domestic energy bills and road fuel and other petroleum product bills.
To find out what those prices are you have to buy the data from someone like Experian, from something like Experian’s Catalist fuel price data – daily site retail fuel prices – data product. You may be able to caluclate the DECC statistics from that data (or you may not) but you certainly can’t go the other way, from the DECC statistics to anything like the Experian data.
But can you go into your local library and ask to look at a copy of the Experian data? A copy of the data that may or may not be used to generate the DECC road fuel and other petroleum product price statistics (how do they generate those statistics anyway? What raw data do they use to generate those statistics?)
Can you imagine ant-eye-ant-eye-consumer data sets being published by your local council or your county council or your national government that can be used to help you hold companies to account and help you tell them that you know they’re ripping you off and your council off and your government off and that together, you’re not going to stand for it?
Can you imagine your local council publishing the forecourt fuel prices for one petrol stations, just one petrol station, in your local council area every day? And how about if they do it for two petrol stations, two petrol stations, each day? And if they do it for three forecourts, three, can you imagine if they do it for three petrol stations…? And can you, can you imagine prices for 50 petrol stations a day being published by your local council, your council helping you inform yourself about how you’re being manipulated, can you imagine…? (It may not be so hard – food hygiene ratings are published for food retail environments across the England, Northern Ireland and Wales…
So let’s hear it for open data, and how open data can be used to hold corporates to account, and how public bodies can use open data to help you make better decisions (which is a good neoliberal position to take and one which the other folk on the bench tell you that that’s what you want and that and markets work, though they also fall short of telling you that the models say that markets work with full information but you don’t have the information, and even if you did, you wouldn’t understand it, because you don’t really know how to make a good decision, but at the end of the day you don’t want a decision, you just want a good service fairly delivered, but they don’t tell that it’s all right to just want that…)
And let’s hear it for public bodies making data available whether it’s open or not, making it available by paying for it if they have to and making it available via library services so that we can start using it to start holding companies to account and start helping our public services, and ourselves, protect ourselves from the attacks being mounted on us by companies, and their national government supporters, who take on debt, and who allow them to take on debt, to make dividend payouts but not capital investment and subsidise the temporary driving down of prices (which is NOT a capital investment) through debt subsidised loss leading designed to crush competition in a last man standing contest that will allow monopolistic last man standing price hikes at the end of it…
And just remember, if there’s anything you want, you know where you can get it… At Alice’s… or the library… only they’re shutting them down, aren’t they…? So that leaves what..? Google?
[Seems I forgot to post this, though I started drafting it on May 19th... Anyway, things seem to have moved on a bit...]
A search related story in the news last week reported on a ruling by the European Union Court of Justice that got wide billing as a “right to be forgotten” (eg BBC News: EU court backs ‘right to be forgotten’ in Google case).
Here’s another example of “censorship”? WordPress not allowing me to link to a URL because it insists on rewriting the & characters in it – here’s the actual link:
For stories like this, I try to look at the original ruling but also tend to turn to law blogs such as my colleague Ray Corrigan’s B2fxxx (though he hasn’t posted on this particular story yet?) or Pinsent Mason’s Out-law (eg Out-law: Google data protection ruling has implications for multi-faceted global businesses) to find out what was actually said.
Here’s the gist of the rulings:
- “the activity of a search engine consisting in finding information published or placed on the internet by third parties, indexing it automatically, storing it temporarily and, finally, making it available to internet users according to a particular order of preference must be classified as ‘processing of personal data’ within the meaning of Article 2(b) when that information contains personal data and, second, the operator of the search engine must be regarded as the ‘controller’ in respect of that processing, within the meaning of Article 2(d).” So what? A person has a right to object to a data controller about the way their data is processed and can obtain “the rectification, erasure or blocking of data” because of its “incomplete or inaccurate nature”.
- “processing of personal data is carried out in the context of the activities of an establishment of the controller on the territory of a Member State, within the meaning of that provision, when the operator of a search engine sets up in a Member State a branch or subsidiary which is intended to promote and sell advertising space offered by that engine and which orientates its activity towards the inhabitants of that Member State.” So Google was found to be “established” in EU member state territories. Are there any implications from that ruling regards tax situation, I wonder?
- Insofar as the processing of personal data that has been subject to a successful objection goes, “the operator of a search engine is obliged to remove from the list of results displayed following a search made on the basis of a person’s name links to web pages, published by third parties and containing information relating to that person, also in a case where that name or information is not erased beforehand or simultaneously from those web pages, and even, as the case may be, when its publication in itself on those pages is lawful.” Note there are limits on this in the case of legitimate general public interest.
- The final ruling seems to at least admit the possibility that folk can request data be taken down without them having to demonstrate that it is prejudicial to them? “when appraising the conditions for the application of those provisions, it should inter alia be examined whether the data subject has a right that the information in question relating to him personally should, at this point in time, no longer be linked to his name by a list of results displayed following a search made on the basis of his name, without it being necessary in order to find such a right that the inclusion of the information in question in that list causes prejudice to the data subject. As the data subject may, in the light of his fundamental rights under Articles 7 and 8 of the Charter, request that the information in question no longer be made available to the general public on account of its inclusion in such a list of results, those rights override, as a rule, not only the economic interest of the operator of the search engine but also the interest of the general public in having access to that information upon a search relating to the data subject’s name. “However, that would not be the case if it appeared, for particular reasons, such as the role played by the data subject in public life, that the interference with his fundamental rights is justified by the preponderant interest of the general public in having, on account of its inclusion in the list of results, access to the information in question.”.
[Update, July 2nd, 2014]
It seems as if things have moved on – Google is publishing notices in the google.co.uk territotry at least to the effect that “Some results may have been removed under data protection law in Europe” [my emphasis].
The FAQ describes what’s happening thus:
When you search for a name, you may see a notice that says that results may have been modified in accordance with data protection law in Europe. We’re showing this notice in Europe when a user searches for most names, not just pages that have been affected by a removal.
The media are getting uppity about it of course, eg Peston completely misses the point, as well as getting it wrong?
In fact, it seems as if the BBC themselves are doing a much better job of obliviating Peston from their own search results…
What all the hype this time around seems to be missing – as with the reporting around the original ruling – is the interpretation that the court ruled on about the behaviour of the search engines insofar as they are deemed to be processors of “personal data”. (Of course, these companies also process personal data as part of the business of operating user accounts, but the business functions – of managing user accounts versus operating a search index and returning search results from public queries applied to it – are presumably sandboxed as far as data protection legislation goes.)
If Google is deemed to be a data controller of personal data that is processed as a result of the way it operates its search index, it presumably means that I can make a subject access request about the data that the Google search index holds about me (as well as the subject access requests I can make to the bit of Google that operates the various Google accounts that I have registered).
As far as the loss of the “right to discover” that the hacks are banging on about as a consequence of “the right to be forgotten”, does this mean that Google is the start and end point of their research activity? (And also putting aside the point that most folk: a) don’t look past the first few results; b) are rubbish at searching. As far as search engine ranking algorithms go – erm, what sort of “truth” do you think reveal? How do you think Google ranks results? And how do you think it comparatively ranks content generated years ago (when links were more persistent than a brief appearance in Twitter feeds and Facebook streams) to content generated more recently (that doesn’t set up persistent link structures)?)
Don’t they use things like Nexis UK?
Or if anything other than Google is too hard, they can just edit the URL to use google.com rather than google.co.uk…
This is where it probably also starts to make sense to look back to the original ruling and spend some time reading it more closely. Is LexisNexis a data controller, subject to data protection legislation, based on it’s index of news media content? Are the indices it operates around court cases similarly covered?
Regularly Scheduled FOI Requests as a None Too Subtle Regular OpenData Release Request? And Some Notes on Extending FOI
A piece of contextualisation in an interview piece with Green MP Caroline Lucas in Saturday’s Guardian (I didn’t do this because I thought it was fun) jumped out at me as I read it: More than 50 energy company employees have been seconded to the government free of charge, and dozens of them to the department of energy and climate change.
How about the gov.uk site?
(I don’t know what’s going in the fight between GDS and the data.gov.uk folk ito getting central gov #opendata info discoverable on the web, but the http://www.gov.uk domain seems to be winning out, not least because for departments who’re in that empire, that’s where any data that eventually linked to from data.gov.uk will actually be published?)
So – it seems folk have been FOIing this sort of information, but it doesn’t look as if this sort of information is being published according to a regular schedule under an #opendata transparency agenda.
Anyone would thing that the UK government wasn’t in favour of a little bit of light being shone on lobbying activity…
I guess I could put a request in to the ODUG (Open Data User Group) for this data to be released as open data, but my hunch is it’s not really the sort of thing they’re interested in (I get the feeling they’re not about open data for transparency, but (perhaps unfairly…?!) see them more as a lobbying group (ODUG membership) for companies who can afford to buy data but who would rather the tax payer pays for its collection and the government then gifts it to them).
More direct would be to find a way of automating FOI requests using something like WhatDoTheyKnow that would fire off an FOI request to each central government department once a month asking for that previous months’ list of secondments into and out of that department in the preceding month (or in the month one or two months preceding that month if they need a monthly salary payment cycle or two for that data to become available).
Of course, it does seem a bit unfair that each government department should have to cover the costs of these requests, but as it stands I can’t make an FOI request of companies that choose to engage in this sort of presumably public service.
Making private companies offering public services under contract subject to FOI does seem to be on the agenda again though, after being knocked back around this time last year?:
An extension to the scope of the FOI Act was proposed a few weeks ago in the Public Bill Committee debate of the morning of Tuesday 18 March 2014 on the Criminal Justice & Courts Bill, columns 186-193:
Dan Jarvis: I beg to move amendment 37, in clause 6, page 6, line 29, at end insert—
‘(1A) The Code of Practice must include a requirement that a person carrying out electronic monitoring who is not a public authority as defined by section 3 of the Freedom of Information Act 2000 shall provide information in respect of the carrying out of electronic monitoring in the same manner as if they were such a public authority.’.
The Chair: With this it will be convenient to discuss amendment 38, in schedule 4, page 73, line 25, at end insert—
‘(1A) Where the Secretary of State enters into a contract with another person under paragraph 1(1), and that person is not a public authority for the purposes of section 3 of the Freedom of Information Act 2000, that person shall be designated by the Secretary of State as a public authority for the purposes of that section in relation to that contract.’.
I remind the Committee that this group is about freedom of information provisions as they apply to aspects of the Bill. Members will have the opportunity to debate the detail of secure colleges later.
Dan Jarvis: I serve notice that, unless sufficient assurances are received, we intend to put the amendments to a vote. [ Interruption. ] Dramatic! I sensed for a moment that there was a higher authority raising a concern about these amendments, but I shall plough on regardless, confident in the knowledge that they are true and right.
Anyone who knows the story of Jajo the rabbit will understand what I am about to say. For those members of the Committee who do not know, Jajo was the pet rabbit successfully registered as a court translator and then booked in for shifts following the Ministry of
Column number: 187
Justice’s outsourcing of language service contracts. Jajo’s short-lived translation career says less about his talent and much more about the importance of ensuring that public contracts delivered by private providers are properly managed.
As was touched on, Ministers now have to manage another fall-out. Two private providers of electronic monitoring overcharged the taxpayer by millions of pounds for tagging offenders who had died or moved abroad, or who were already back in prison. That underlines the case for the amendments.
Both amendments would seek to bring non-public providers of public services contracted out under the Bill within the scope of the Freedom of Information Act. Amendment 37 relates to clause 6 and the code of practice that would be issued by the Secretary of State on the processing of data related to electronic monitoring. It would require anyone carrying out monitoring related to the clauses to comply with FOI requests in the same way as public bodies do. Amendment 38 relates to schedule 4 and the arrangements for contracting out secure colleges, which are detailed in part 2. It would require anyone contracted to provide a secure college to comply with freedom of information in the same way. Both our proposals are worthy of consideration by the Committee.
We all know that the landscape of how public services are delivered is changing. The Government spend £187 billion on goods and services with third parties each year, about half of which is estimated to be on contracting out services. About half of all spending on public services now ends up in private providers’ hands and more and more private providers are bidding to take on the responsibility and financial rewards that come with large-scale public contracts. As outsourcing is stepped up, more and more information about public services and public money is being pulled out of the public domain. That presents a particular challenge that we must tackle.
As the Information Commissioner told the Justice Committee last year,
“if more and more services are delivered by alternative providers who are not public authorities, how do we get accountability?”
The rewards that third parties stand to gain need to go hand in hand with the duties of transparency and information sharing. The public should be able to ask about how, and how well, the service they are paying for is being run.
The Freedom of Information Act does provide for supply-chain companies to be considered to be holding information on behalf of a public authority. In practice, however, contracted providers in the justice sector are not subject to anywhere near the same transparency requirements as publicly-run services. Private prisons, for example, are not subject to FOI in the same way as public prisons and the experience of G4S, Serco and others will have influenced many other companies not to be as forthcoming as they might have been. That is why we need to build freedom of information into the contracts that the Government make with third parties.
The Committee will be aware that such an approach was recommended by the Public Accounts Committee in its excellent report published last week. It made the
Column number: 188
point that many Departments are not providing information on how those contracts work on the grounds of commercial confidentiality. The public will not accept that excuse for much longer.
Let me conclude my remarks by offering the Committee a final quote. Someone once said:
“Information is power. It lets people hold the powerful to account”
and it should be used by them to hold their
“public services to account”.
I agree with the Prime Minister. Two years ago, he spoke about
“the power of transparency”
“why we need more of it.”
He also spoke of leading
“the most transparent Government ever.”
Labour has pledged that the next Labour Government will deal with the issue by bringing companies providing public contracts into the scope of FOI legislation.
Freedom of information can be uncomfortable. It can shed light on difficult issues and be problematic for Government Ministers, but that is the point. The Committee has the opportunity today to improve the Bill and to get a head start.
Dr Huppert: I will not detain the Committee. I share the concern about the lack of FOI for private organisations providing public services. My colleagues and I have expressed concerns about that for many years, and the previous Government were not very good at accepting that. It is good news that the Labour party may undo that error.
Mr Slaughter: Can the hon. Gentleman say what steps he and the coalition have taken to extend FOI in the past four years?
Dr Huppert: Not as many as I would like, but we have seen progress in some areas; we did not see any at all when the hon. Gentleman was a Minister. I hope we will see the correct drive. I share the concern that we need transparency when public services are delivered by private companies. They must not be shielded. I look forward to hearing what the Minister has to say, because he has commented on such issues before.
It is important that the matter should be dealt with on a global scale. I think the shadow Minister would agree that the case is broader. I hope to hear from the Minister that there will be more work to look at how the issue can be addressed more generally, rather than just in a specific case. That would probably require amendment of the Freedom of Information Act. That is probably the best way to resolve the issue, rather than tacking it on to this area, but I absolutely share the concerns. I hope we can see more transparency, both from the Government—we are seeing that—and from the private sector as it performs public functions.
Yasmin Qureshi: The Justice Committee, of which I am a member, looked into the Freedom of Information Act and how it has been operating since it was passed many years ago. We spoke to different groups of people,
Column number: 189
representatives of councils and local authorities, the Information Commissioner and pressure groups. Generally, the view was that the Freedom of Information Act has been a force for good. The thing that people mentioned time and again was the fact that it applies only to public authorities and has a narrow remit in private companies. A lot of concern was expressed about that.
As my hon. Friend the Member for Barnsley Central said, just under £200 billion is being spent by the Government for private companies to carry out public work. The number of outsourcings could increase, especially in the criminal justice system. In the probation service there will be contracting out and privatisation, as well as changes in the criminal justice system in relation to legal aid and suchlike. We have concerns about the criminal justice system and the number of companies that will be carrying out work that the state normally does. It is an important issue.
Will the Minister give us an undertaking for whenever Government money is given to carry out work on behalf of the Government? Local authorities and Government Departments have to provide information, and it should be the same for private companies. At the moment, as the shadow Minister mentioned, the agencies providing some of the public work give some information, but it is not enough.
It is often hard to get information from private companies. It is important for the country that we know where public money is being spent and how private companies respond to such things. We can have party political banter, but freedom of information was introduced many years ago and has been working well. Freedom of information needs to be extended in light of the new circumstances. I ask for a clear commitment from the Government that they will encapsulate that in the Bill. They now have that opportunity; the Labour party has said that, if it was in government, it would certainly do so. The lacunae and the gaps would be addressed by the amendment, which would make it clear exactly how the regime applies. [Interruption.]
The Chair: I apologise for the background noise. We are looking into the cause.
Jeremy Wright: Thank you, Mr Crausby. I hope Jajo the rabbit is not responsible.
As the hon. Member for Barnsley Central said, amendment 37 seeks to introduce a requirement as to the contents of the code of practice that the Secretary of State will issue under proposed new section 62B of the Criminal Justice and Court Services Act 2000, which is to be introduced through clause 6. The Secretary of State would have to include provisions in the code of practice requiring providers of outsourced electronic monitoring services to make information available in the same manner as if they were subject to the provisions of the Freedom of Information Act. The aim of the amendment seems essentially to extend the Act to providers of electronic monitoring not already subject to its provisions.
Amendment 38 has the same basic intention in that it seeks to extend the Freedom of Information Act to providers of secure colleges that have entered a contract with the Secretary of State to do so under schedule 4. The approach differs, however, because amendment 38
Column number: 190
would extend the Act directly, whereas amendment 37 seeks to extend its obligations through code of practice guidance.
In other words, both amendments would require private providers not currently subject to the Freedom of Information Act to make information available both in response to FOI requests and proactively through publication schemes. Section 5 of the Act already provides a power to extend the Act’s provisions to contractors providing public services. For reasons I will try to outline, the Government do not currently propose to adopt that approach and are adopting an alternative method to ensure transparency. I am aware, however, of the long-standing and serious concerns raised on the position under the Act of private providers of public services. It might help the hon. Member for Hammersmith to know that the Government are committed to, and have taken steps to extend, the Act. More than 100 additional organisations have been included since 2010, and we are considering other ways in which its scope may be widened.
The issue of outsourced public services was considered in some detail during post-legislative scrutiny of the Freedom of Information Act by the Select Committee on Justice in 2012. I do not know whether the hon. Member for Bolton South East was a member of the Committee of that time, but the Committee rightly issued a reminder that
“the right to access information is crucial to ensuring accountability and transparency for the spending of taxpayers’ money”.
The Committee recommended the use of contractual provisions, rather than the formal extension of the Act, to ensure that transparency and accountability are maintained. In particular, the Committee said:
“We believe that contracts provide a more practical basis for applying…outsourced services than partial designation of commercial companies under section 5 of the Act”.
The Committee also feels that
“the use of contractual terms to protect the right to access information is currently working relatively well.”
The Government’s approach is consistent with that recommended by the Justice Committee.
In addition to information being made available proactively, the Government are taking steps to issue a revised code of practice under section 45 of the Freedom of Information Act to promote transparency on outsourced public services in response to FOI requests. The code of practice will be issued later this year and will promote and encourage the use and enforcement of contractual obligations to ensure that private bodies not subject to the Act provide appropriate assistance where information about outsourced public services is requested from bodies that are subject to the Act.
The Government recognise that only a small amount of information held by private providers is currently often formally subject to the Act. Our code of practice will encourage public authorities to go further, to interpret their freedom of information obligation broadly and to release more information on a voluntary basis, where it would be in the public interest to do so. In the event of non-compliance, it will also be possible for the Information Commissioner to issue and publish a practice recommendation setting out steps that, in his view, the public authority should take to promote conformity with the guidance.
Column number: 191
Mr Slaughter: I seem to remember taking part in the Westminster Hall debate arising out of the Justice Committee’s deliberation and I do not think that it was very happy with the approach that the Government are taking, particularly where they are seeking to restrict freedom of information further. Does the hon. Gentleman accept on the basis of what he has just said that this will not be a level playing field and that the same requirements that apply to public bodies will not apply to private organisations undertaking an effectively identical role? Does he accept that, whatever the merits of his scheme, it does not to far enough and does not address the comments of my hon. Friend the Member for Barnsley Central?
Jeremy Wright: The hon. Gentleman will recognise that the organisations we are talking about extending the provisions of the Act to cover vary hugely in size and level of resources. The concern is to draw the appropriate balance between giving correct access to information and not imposing intolerable burdens on organisations, particularly smaller ones. That is the balance that has to be struck. We are looking at ways in which we can continue to make public authorities responsible for supplying information but ensure that it comes from the place where it originated, which may be those other organisations.
Mr Slaughter: That is a different argument and one that is often tried. It was tried in relation to universities and to the smaller district councils much beloved of the hon. Member for Bromley and Chislehurst. There are already limitations within the Act. There are safeguards for organisations in terms of the amount of time and cost. Why are they not sufficient?
Jeremy Wright: As I said, there is a balance to be struck. We attempt to strike that balance correctly with our proposals. If I can explain what we want to do a little more fully, perhaps the hon. Gentleman will be reassured—although frankly I doubt it. There is an opportunity for us to look at the issue in a sensible way with the code of practice. Applying our forthcoming code of practice guidance across the public sector will ensure that transparency and response to freedom of information requests will be maintained in a consistent way. This is preferable—I agree with my hon. Friend the Member for Cambridge—to the more piecemeal approach promoted by amendments 37 and 38.
The success of our own code of practice will be monitored by the Ministry of Justice and the Information Commissioner. We were clear in our response to post-legislative scrutiny of the Freedom of Information Act that, should this approach yield insufficient dividends, we will consider what other steps are necessary. In summary, we are committed to ensuring transparency in relation to all outsourced public services, including electronic monitoring and secure colleges. We are taking steps to ensure that through the code of practice to be issued later this year. On that basis, I invite the hon. Gentleman to withdraw his amendment.
Yasmin Qureshi: The Minister referred to the Select Committee on Justice and its recommendations. As you know, without going into the detail of that discussion, Select Committee recommendations sometimes tend to
Column number: 192
be compromises. At the time, three issues were in the mind of the Select Committee. First, it did not realise that a legislative opportunity would come so soon in which to put the measure in a more codified way with a clearer legal obligation. Secondly, there was quite a lot of discussion about private companies.
The Select Committee accepted that the Freedom of Information Act should not apply to purely private companies carrying out purely private work; it was not really arguing against that. However, here we have an opportunity to codify once and for all in legislation the provision that the FOIA should apply whenever public money is paid to a private company to carry out work. That would be a fairly straightforward provision. I do not see why we need to go down the complicated route of using a code of practice, putting in a specific provision in a new contract each time something happens. Why can we not just have a general provision that applies to every situation?
Jeremy Wright: I was a member of the Justice Committee before the hon. Lady was, so I understand her point that recommendations of the Select Committee are a matter of discussion and compromise. However, they are made on a cross-party basis, and paid all the more attention to for that reason. I quoted directly from the Select Committee’s conclusions in what I said earlier.
On the hon. Lady’s other point, this may be an earlier legislative opportunity than the Select Committee anticipated, but of course, it is only an opportunity in relation to specific policies. Again, I rather agree with the point made earlier by my hon. Friend the Member for Cambridge: there is an argument for addressing the issue, not on a piecemeal basis, but more comprehensively.
The hon. Lady’s final point is that the approach that we have set out—using a code of practice—is inadequate and that a statutory approach should be introduced by amending primary legislation. An initial approach of using a code of practice is a sensible one. She will recognise that amendment 37, tabled by the hon. Member for Barnsley Central, deals with a requirement in a code of practice, not primary legislation. Amendment 38 is different, but in relation to electronic monitoring, on which a number of concerns have been expressed, the hon. Gentleman’s chosen vehicle is a code of practice. The code of practice approach appears to be welcomed by both sides of the Committee.
Dan Jarvis: I have listened carefully to the Minister’s response. Clearly, we will want to look carefully at the detail of what he has said about a code of practice.
I agree with my hon. Friend the Member for Bolton South-East that the Committee has an opportunity this morning to make progress on redefining the freedom of information. I have heard the Minister’s response to that point, but the reality is that the move would be popular with the public.
There is no doubt that the landscape in which public services are delivered is changing. The Opposition have pledged to reform freedom of information if we are in government from 2015. I am mindful of the Prime Minister’s comments, which I quoted earlier. He said:
Column number: 193
“Information is power. It lets people hold the powerful to account”,
and it should be used by them to hold their public services to account.
Mike Kane: Does my hon. Friend agree that, as the contracting out of public services expands, the public’s right to information shrinks?
Dan Jarvis: I agree absolutely. There is a degree of inevitability that we will see change in the area. The debate is about how we do it, and it is important that we have that debate. We have tabled the amendments partly so that we can take the opportunity to debate such issues.
Mr Slaughter: There is another point here, which is that the Ministry of Justice is particularly vulnerable on the issue. We have had the privatisation of the probation service and the scandals regarding tagging. We will come to later in the Bill to proposals about the externalisation of the collection of fines and other matters. First, that is going on wholesale in the Department, and secondly, it is defective in many aspects. It is particularly relevant that the Minister should accept that the proposals in the Bill are not sufficient.
Dan Jarvis: My hon. Friend is right. In the context of the delivery of public services within the Ministry of Justice remit, this is a particularly relevant, timely and important issue. It has been incredibly useful to have the opportunity to debate it owing to the tabling of the amendments.
I mentioned that I was mindful of the Prime Minister’s comments, and I am mindful of the fact that the Justice Secretary has also indicated a desire to reform freedom of information. Given that there is a general acknowledgment that the status quo is not acceptable and despite what the Minister has said in response to our amendment, I will press it to a vote.
The amendment was defeated.
An hour or so later, the government took this line:
Freedom of Information Act
23. Lindsay Roy (Glenrothes) (Lab): What plans he has to bring forward legislative proposals to expand the scope of the Freedom of Information Act 2000.
The Minister of State, Ministry of Justice (Simon Hughes): There has been good progress in extending the implementation of the Freedom of Information Act because the coalition Government pledged to extend its scope to provide greater transparency. We extended it in 2010 to academies, in 2011 to the Association of Chief Police Officers, the Financial Ombudsman Service and the Universities and Colleges Admissions Service, and last year to 100 companies wholly owned by more than one public authority. The next item on the agenda is to do with Network Rail, and we are awaiting a view from the Department for Transport as to whether it thinks it would be appropriate for that to be implemented this year.
Lindsay Roy: What benefits have accrued to the Government and citizens from the implementation of the Act, and when does the Minister plan to extend its scope further?
Simon Hughes: We intend to extend it further as soon as is practical. One specific issue that I hope will be of interest to the hon. Gentleman—as it is to colleagues of his, including those who have come to see me about it—is that we intend to publish a revised code of practice to make sure that private companies that carry out public functions have freedom of information requirements in their contracts and go further than that. We hope that that will be in place by the end of this year.
Mr Mark Harper (Forest of Dean) (Con): There is one area where the Minister should perhaps look at narrowing the scope of the Act, because my understanding is that requests can be made by anybody anywhere on the face of the earth; they do not have to be British citizens. It is not the role of the British Government to be a taxpayer-funded research service for anyone on the globe. May I suggest that he narrow the scope to those for whom the Government work—citizens of our country?
Simon Hughes: I well understand my hon. Friend’s point. There will be two consultations this year: first, on precisely such issues about the scope of the current legislation to make sure that it is not abused while we retain freedom of information as a principle of Government; and secondly, on extending it to other areas where we have not gone so far.
Dr Huppert:I read out the quote from someone who has made the position clear when it comes to private companies carrying out public functions. Indeed, the code of practice has exactly the wording used in amendment 11, which the hon. Gentleman supported when we debated it on Tuesday. I do not want to take up too much of the Chairman’s kindness to discuss an issue that was rejected at that point, but it is happening as we wanted.
The matter was also touched upon a couple of days later in a Public Bill Committee on the Criminal Justice and Courts Bill (Official Report, Thursday 20 March 2014, 257-259) where accountability around public contracts delivered by private provides was being discussed:
Mr Slaughter: Absolutely not. I hope that the hon. Gentleman has read the article about Jago the rabbit that my hon. Friend the Member for Barnsley Central (Dan Jarvis) and I wrote for The Independent yesterday [It’s time we extended Freedom of Information to public services run by private companies – just ask Jago the Rabbit], which dealt with what should be done, which is to bring these companies within the ambit of FOI, and what the Minister of State did—with his usual skill, shall we say?—at Justice questions on Tuesday. He implied that that was what was going to happen, whereas in fact he was doing nothing more than putting round the line that the Cabinet Office has already indicated.
If I am wrong about that, I will give way in a moment and the hon. Gentleman can come back to me, but my understanding is that the Government—both parts of it, as long as they are just about coalescing—are of the view that the contracts that are drawn up should include this notional transparency. That is to say that they will encourage public authorities to encourage private companies to put clauses into contracts that will expose as much as possible, within the realms of commercial confidentiality. So the contracts will be open, with publication on websites and so forth of as much information about the contract as the two parties think fit. What we will not have is a duty on those private companies—in so far as they are carrying out public functions—to comply with the terms of the Freedom of Information Act, as would be the case in the public sector.
I accept that they are two sides of the same coin. On the one hand, of course it is a good idea that the information is made available voluntarily, but if it is not—either because the company does not choose to do so or because the contract is not drafted sufficiently well to ensure that it must—the citizen must have the right, through FOI, to require that information to be made available. As far as I am concerned, that is not what was said on Tuesday. I know that there is consultation going on, but if it is the intention of the Government—at least the Liberal Democrat part of the Government—to follow the line taken by my right hon. Friend the Member for Tooting (Sadiq Khan), the shadow Lord Chancellor, which he has repeated often in recent months, and require all those private companies performing public functions to come within the requirements of the Freedom of the Information Act, I would be pleased if the hon. Gentleman said so now.
Mr Slaughter:I take from that comment that even the hon. Gentleman does not understand what the Minister of State, Ministry of Justice, the right hon. Member for Bermondsey and Old Southwark, says, so opaque is it. If nobody, including the Minister, is going to answer my question, the answer will no doubt come out in the wash on a later occasion. However, it seems to me that that is not what is being promised. If it were, the Minister would be jumping up and claiming credit for it, but he is not. Therefore, I assume that that is not the case.
The significance of that is that those four companies about which I have just raised doubts—G4S, Serco, Capita, and we can safely add Atos—all told the Public Accounts Committee that they were prepared to accept the measures that the Committee proposed. It therefore appears that the main barrier to greater transparency lies within Government.
That is where we are. Even the companies that want to put themselves and the interests of their shareholders first are more keen on transparency and on answering the legitimate questions that are being asked by everyone— from ourselves to the chief inspector of prisons—than this Government are.
I say that because if we are to take this further leap down that path, it is only right that the Government do not just challenge, as the Minister has said, acknowledged frauds, but look at the entire performance behaviour, as well as the number of available companies that could step into the breach and deal with these matters.
What we must conclude from the conjunction of clauses 17 and 18 is that, first, the Government are prepared to take this leap in the dark, in terms of the reconfiguration of the youth estate and, secondly, that they are prepared to leave that entirely in the hands of the people who failed so many times in so many contracts, not least in running parts of the adult prison service.
For more on some of the specifics, see the House of Commons Public Accounts Committee report on “Contracting out public services to the private sector”, which for example recommended “that the Cabinet Office should explore how the FOI regime could be extended to cover contracts with private providers, including the scope for an FOI provision to be included in standard contract terms; that neither the Cabinet Office nor departments should routinely use commercial confidentiality as a reason for withholding information about contracts with private providers; [and that] The Cabinet Office should set out a plan for departments to publish routinely standard information on their contracts with private providers”.
There’s also a couple of related private members bills floating around at the moment – Grahame Morris’ empty Freedom of Information (Private Healthcare Companies) Bill 2013-14 “to amend the Freedom of Information Act 2000 to apply to private healthcare companies”, and Caroline Lucas’ Public Services (Ownership and User Involvement) Bill 2013-14 “to put in place mechanisms to increase the accountability, transparency and public control of public services, including those operated by private companies”. The latter >a href=”http://www.publications.parliament.uk/pa/bills/cbill/2013-2014/0160/cbill_2013-20140160_en_2.htm#l1g5″>proposes:
(1) Where a relevant authority starts the process of procurement for a public services contract, it must make available to the public details of all bids received prior to the conclusion of the procurement exercise.
(2) Where a relevant authority enters into a public services contract, details of that contract shall be made available to the public within 28 days of the procurement decision.
6 Freedom of information
(1) The Secretary of State must designate as a public authority, pursuant to section 5(1)(b) of the Freedom of Information Act 2000, companies or other bodies which enter into a public services contract.
(2) “Public services contract” has the meaning contained within section 8 of this Act.
(3) The Secretary of State shall maintain a list of companies designated under section 6(1) of this Act.
(4) Requests under the Freedom of Information Act 2000 in respect of such companies or bodies can only be made in respect of information relevant to the provision of a public services contract.
(5) The Secretary of State must designate as a public authority, pursuant to section 5(1)(b) of the Freedom of Information Act 2000, any utility company subject to regulation by regulatory authorities as defined in section 8.
Finally, on the accountability and transparency thing, there’s a consultation on at the moment regrading “smaller authorities with an annual turnover not exceeding £25,000, including parish councils, [who] will be exempt from routine external audit” but instead will be subject to a transparency code (Draft transparency code for parish councils – consultation).
Related: Spending & Receipts Transparency as a Consequence of Accepting Public Money? If you accept public money for contracts that would otherwise be provided by a public service you should be subject to the same levels of FOI and transparency reporting. Why should public services have to factor this in to their bids for running a service when private companies don’t?
So it seems that in a cost-recovered data release that was probably lawful then but possibly wouldn’t be now* – Hospital records of all NHS patients sold to insurers – the
Staple Inn Actuarial Society Critical Illness Definitions and Geographical Variations Working Party (of what, I’m not sure? The Institute and Faculty of Actuaries, perhaps?) got some Hospital Episode Statistics data from the precursor to the HSCIC, blended it with some geodemographic data**, and then came to the conclusion that “that the use of geodemographic profiling could refine Critical illness pricing bases” (source: Extending the Critical Path), presenting the report to the Staple Inn Actuarial Society who also headline branded the PDF version of the report? Maybe?
* House of Commons Health Committee, 25/2/14: 15.59:32 for a few minutes or so; that data release would not be approved now: 16.01:15 reiterated at 16.03:05 and 16.07:05
** or maybe they didn’t? Maybe the data came pre-blended, as @amcunningham suggests in the comments? I’ve added a couple of further questions into my comment reply… – UPDATE: “HES was linked to CACI and Experian data by the Information Centre using full postcode. The working party did not receive any identifiable data.”
CLARIFICATION ADDED (source )—-
“In a story published by the Daily Telegraph today research by the IFoA was represented as “NHS data sold to insurers”. This is not the case. The research referenced in this story considered critical illness in the UK and was presented to members of the Staple Inn Actuarial Society (SIAS) in December 2013 and was made publically available on our website.
“The IFoA is a not for profit professional body. The research paper – Extending the Critical Path – offered actuaries, working in critical illness pricing, information that would help them to ask the right questions of their own data. The aim of providing context in this way is to help improve the accuracy of pricing. Accurate pricing is considered fairer by many consumers and leads to better reserving by insurance companies.
There was also an event on 17 February 2014.
Via a tweet from @SIAScommittee, since deleted for some reason(?), this is clarified further: “SIAS did not produce the research/report.”
The branding that mislead me – I must not be so careless in future…
Many of the current agreements about possible invasions of privacy arising from the planned care.data release relate to the possible reidentification of individuals from their supposedly anonymised or pseudonymised health data (on my to read list: NHS England – Privacy Impact Assessment: care.data) but to my mind the
SIAS report presented to the SIAS suggests that we also need to think about consequences of the ways in which aggregated data is analysed and used (for example, in the construction of predictive models). Where aggregate and summarised data is used as the basis of algorithmic decision making, we need to be mindful that sampling errors, as well as other modelling assumptions, may lead to biases in the algorithms that result. Where algorithmic decisions are applied to people placed into statistical sampling “bins” or categories, errors in the assignment of individuals into a particular bin may result in decisions being made against them on an incorrect basis.
Rather than focussing always on the ‘can I personally be identified from the supposedly anonymised or pseudonymised data’, we also need to be mindful of the extent to, and ways in, which:
1) aggregate and summary data is used to produce models about the behaviour of particular groups;
2) individuals are assigned to groups;
3) attributes identified as a result of statistical modelling of groups are assigned to individuals who are (incorrectly) assigned to particular groups, for example on the basis of estimated geodemographic binning.
What worries me is not so much ‘can I be identified from the data’, but ‘are there data attributes about me that bin me in a particular way that statistical models developed around those bins are used to make decisions about me’. (Related to this are notions of algorithmic transparency – though in many cases I think this must surely go hand in hand with ‘binning transparency’!)
That said, for the personal-reidentification-privacy-lobbiests, they may want to pick up on the claim in the
SIASIFoA report (page 19) that:
In theory, there should be a one to one correspondence between individual patients and HESID. The HESID is derived using a matching algorithm mainly mapped to NHS number, but not all records contain an NHS number, especially in the early years, so full matching is not possible. In those cases HES use other patient identifiable fields (Date of Birth, Sex, Postcode, etc.) so imperfect matching may mean patients have more than one HESID. According to the NHS IC 83% of records had an NHS number in 2000/01 and this had grown to 97% by 2007/08, so the issue is clearly reducing. Indeed, our data contains 47.5m unique HESIDs which when compared to the English population of around 49m in 1997, and allowing for approximately 1m new lives a year due to births and inwards migration would suggest around 75% of people in England were admitted at least once during the 13 year period for which we have data. Our view is that this proportion seems a little high but we have been unable to verify that this proportion is reasonable against an independent source.
Given two or three data points, if this near 1-1 correspondence exists, you could possibly start guessing at matching HESIDs to individuals, or family units, quite quickly…
- ACORN (A Classification of Residential Neighbourhoods) is CACI’s geodemographic segmentation system of the UK population. We have used the 2010 version of ACORN which segments postcodes into 5 Categories, 17 Groups and 57 Types.
- Mosaic UK is Experian’s geodemographic segmentation system of the UK population. We have used the 2009 version of Mosaic UK which segments postcodes into 15 Groups and 67 Household Types.
The ACORN and MOSAIC data sets seem to provide data at the postcode level. I’m not sure how this was then combined with the HES data, but it seems the
SIASIFoA folk found a way (p 29) [or as Anne-Marie Cunningham suggests in the comments, maybe it wasn't combined by SIASIFoA - maybe it came that way?]:
The HES data records have been encoded with both an ACORN Type and a Mosaic UK Household Type. This enables hospital admissions to be split by ACORN and Mosaic Type. This covers the “claims” side of an incidence rate calculation. In order to determine the exposure, both CACI and Experian were able to provide us with the population of England, as at 2009 and 2010 respectively, split by gender, age band and profiler.
This then represents another area of concern – the extent to which even pseudonymised data can be combined with other data sets, for example based on geo-demographic data. So for example, how are the datasets actually combined, and what are the possible consequences of such combinations? Does the combination enrich the dataset in such a way that makes it easier for use to deanonymise either of the original datasets (if that is your primary concern); or does the combination occur in such a way that it may introduce systematic biases into models that are then produced by running summary statistics over groupings that are applied over the data, biases that may be unacknowedged (to possibly detrimental effect) when the models are used for predictive modelling, pricing models or as part of policy-making, for example?
Just by the by, I also wonder:
- what data was released lawfully under the old system that wouldn’t be allowed to be released now, and to whom, and for what purpose?
– are the people to whom that data was released allowed to continue using and processing that data?
– if they are allowed to continue using that data, under what conditions and for what purpose?
– if they are not, have they destroyed the data (16.05:44), for example by taking a sledgehammer to the computers the data was held on in the presences of NHS officers, or by whatever other means the state approves of?
In What Role, If Any, Does Spending Data Have to Play in Local Council Budget Consultations? I started wondering about the extent to which local spending transparency data might play a role in supporting consultation around new budgets.
As a first pass, I’ve popped up a quick application up at http://glimmer.rstudio.com/psychemedia/iwspend2013_14/ [if that's broken, try this one] (shiny code here). You can pass form items in via the URL (except to set the Directorate – oops!), and also search using regular expressions, but at the moment still need to hit the Search button to actually run the search. NOTE – there’s a little bug – you need to hit the Search button to get it to show data; note – selecting All directorates and no filter terms can be a bit slow to display anything…
I’ve started exploring various views over the data, but these need thinking through properly (in particular with respect to finding out views that may actually be useful!)
Hmm… did the budget change directorate?!
Some more views over the suppliers tab – I started experimenting with some tabular views in the suppliers tab too…
This is all very “shiny” of course, but is it useful? From these early glimpses over the data, can you think of any ways that a look at the spending data might help support budget consultations? What views over the data, in particular, might support such an aim, and what sort of stories might we be able to tell around this sort of data?
It seems to be that time of year when my local council is consulting about next year’s budgets, running a series of roadshows (which I missed) backed up by an online consultation (but no briefing or discussion documents?)
The online consultation opens with asking for opinions about the extent to which the council should support various services:
Adult Social Care, Asset & Estates Management, Bereavement Services, Central Support Services (Legal, Finance, ICT, Human Resources, Shared Services, Corporate & Democratic Core), Childrens Social Care, Coastal Management, Community Safety, Concessionary Fares, Coroner, Cowes Chain Ferry, Culture & Heritage, Economic Development, Education Psychology/Special Education Needs, Education Welfare Service, Elections & Land Charges, Environmental Health, Licensing, Trading Standards, Fire & Rescue, Harbours, Highways, Housing, Leisure & Recreation, Libraries, Parking Services, PFI Contract, Planning & Building Control, Public Conveniences, Public Health, Registrars, Revenues & Benefits, Schools Infrastructure/buildings, School Music Service, School Transport, Sports & Play Development, Subsidised Buses, Tourism, Waste Management, Workforce Development, Youth Service
The council also publishes a Budget Book which identifies intended savings in three directorate areas (Economy & Environment, Community Wellbeing & Social Care, Schools & Learning).
Budgets are allocated in different service areas according to the Service Reporting Code of Practice for Local Authorities (SeRCOP):
The Revenue Budget Service Analysis breaks things down further within each service area:
Adult social care
Central Services and Capital Costs
Childrens Social Care
Cultural and related services
Planning and development services
Fire and Rescue Services
Highways and Transport Services
Local spending data
Naive as I am, I started wondering about the extent to which we might be able to map spending items released through the council’s Spending and Finance (Transparency) data releases.
So for example, if we know a service area has a particular annual budget, and we also get to see certain declarations of spend within that are (maybe reconciled to contracts and other procurement information), then we might start to be able to pull together a picture about the range of activities associated with the service area based on the trace shadows those activities throw off in the form of spending items.
If a budget cut is proposed to a service area, and it applies proportionally to the spend, or to particular areas of the spend, we might start to get a feeling about what the practical consequences of the cut might actually be.
So that’s the proposition – but how far can we get when we look at the spending data itself?
Reviewing the spending data to date for the 2013-14 financial year, we see spend associated with the following directorates:
- Childrens Services
- Community Wellbeing & Social Care
- Economy & Environment
- Chief Executive, Schools & Learning
It’s not necessarily immediately obvious how these map on to the services listed in the budget book (for example, is Childrens Social Care in Childrens Services or Community Wellbeing & Social Care?
The spending data released by the Isle of Wight Council also has two other relevant columns: the Service Area and the Expenses Type. Unfortunately, the Service Area descriptions seem to be rather ad hoc, and whilst the Expense Area terms may follow some standard vocabulary, it’s not obvious which, or how these terms are associated with particular summary spending areas that appear in the budget book.
As a very quick example of the sort of thing we can start to look for, here is an example of spends associated with the keyword music appearing in the spending Service Area:
The School Music Service is one of the areas the council consultation asked about, so if there is to be a cut to the funding of this service that impacts on spend, we might get a feel for what levels of savings are possible on transparently declared spend in that area:
So responsibility for the service changed directorate in July?
Here’s the overall cumulative spend:
So the transparently declared spend isn’t that much (the majority of spend – not declared in the spending data – is presumably on salaries?), and what there is declared is mainly on travel. As armchair auditors or budget consultees, I guess this provides us with the question: is just over £10k on staff vehicle mileage over the last 6 months a large amount?
We could possibly also get an estimate of the amount of mileage involved (Isle of Wight council mileage rates). As a lower bound on the mileage, if it was all charged at the highest rate (65p per mile) and all the Staff Vehicle mileage was on miles at that rate, it would come to 10490.62 * 100 / 65.0, or just over 16,100 miles; that is, slightly over 2,500 miles per month, or 100 miles per working day.
But again: is this a large number? When you consider that the School Music Service provides “weekly instrumental and vocal tuition” to “some 3,000 pupils a week in 49 primary, secondary and special schools … from a team of 24 qualified and experienced staff” (via about the School Music Service), it doesn’t seem so much?
(I am mindful that doing such exercises can lead to hatchet jobs around small amounts of money on particular public services, and am quite wary of giving examples as a result… What I do hope to show, though, is how this sort of data investigation does do can encourage you to go looking for other sources of data to help make sense of, and further put into context, the data you do have…)
In terms of adult social care, services provided to “older’ people (presumably, 65 and over) appear to be identified in the budget book headings as one coherent group. So what affect might budget cuts in this area have on spending, based on spending items that might be associated with this area?
If we search for mention of the key term elderly, here’s what we get in accumulated spend to date:
Service.Area sumTot Elderly Mentally Ill Residential Care 3511701.13 Elderly Frail Homecare 2084136.78 Elderly Residential Daycare 1725826.30 Elderly Frail Nursing Island 1579067.00 Elderly Frail Residential Care 1169120.25 Elderly Frail Residential Income 1116792.64 Elderly Frail Nursing - Island 669237.95 Elderly Mentally Ill Nursing Island 558575.44 Elderly Frail Managed Accounts 230445.28 Elderly Mental Ill Nursing - Island 216668.02 Elderly Mentally Ill Homecare 154026.30 Elderly Frail Nursing Mainland 33553.91 Elderly Frail Daycare 23377.70 Elderly Mentally Ill Nursing Mainland 11761.68 Elderly Mental Ill Nursing - Mainland 10243.80 Elderly Mentally Ill Managed Accounts 9257.41 Elderly Mentally Ill Daycare 4549.10 Elderly Frail Nursing - Mainland 4515.84 Elderly Frail Other Care 2711.83 Elderly Mentally Ill Other Care 2550.16 Elderly Frail Personal Budgets 1900.64 Elderly Frail Community Equipment 924.39 Elderly Mentally Ill Personal Budgets 625.24 Elderly Frail Non-Res Income 171.15 Elderly Frail - Day/Other care - Client Contributions 36.50 Elderly Mentally Ill Nursing Income -1510.32 Elderly Frail Nursing Income -2371.82
And here are the main suppliers:
Supplier.Name sumTot ISLAND HEALTHCARE LTD 891433.32 REDACTED PERSONAL DATA 836757.77 SCIO HEALTHCARE LTD 685499.04 SOMERSET CARE LTD 616713.96 LONDON RESIDENTIAL HEALTHCARE 517033.27
We can also get a feel for what the expense types are:
Expenses.Type sumTot Charges from Independent Providers 13504438.81 Regular Respite Care 214573.11 Crisis Support for Carers 17951.37 Fixed Telephones 924.39 Professional Service 356.04 Operational Equipment 36.50 Client Expenses 11.83 Client Contributions -1185.83 Provider Refunds -619211.92
With many challenges facing the provision of homecare more generally, how is funding allocated in service areas containing the keyterm homecare?
Service.Area sumTot Elderly Frail Homecare 2084136.78 Physical Disability Homecare 395358.15 Elderly Mentally Ill Homecare 154026.30 Learning Disability Homecare 123971.95 Learning Disabilities Homecare 120884.88 Mental Health Homecare 117730.46 Dementia Homecare 28467.84 Homecare Reablement 20892.01 Homecare 18-25 year olds 12519.84 Children Young Adults Disability Homecare 2962.08
As I mentioned, I didn’t get to go to any of the budget consultation roadshows. But I do wonder whether we can make use of spending data to help us get a feel for some of the services that budgets deliver via spend with third parties. For example, can we use such information as a view on to how cuts to budgets might play out in terms of spending cuts associated with the withdrawal or reduction of corresponding services?
In order to relate spend to budget items described in the budget book, I think there is still some way to go though. What we need is another column in the spending data that relates to headings described in the budget book. (Even better might be columns relating to SerCOP codes?)
PS Hmm, I wonder… is this the sort of topic that could make for an interesting School of Data style community based data expedition? And waht role might the Isle of Wight Armchair Auditor or OpenSpending play in such a thing?