OUseful.Info, the blog…

Trying to find useful things to do with emerging technologies in open education

Archive for the ‘Policy’ Category

Regularly Scheduled FOI Requests as a None Too Subtle Regular OpenData Release Request? And Some Notes on Extending FOI

A piece of contextualisation in an interview piece with Green MP Caroline Lucas in Saturday’s Guardian (I didn’t do this because I thought it was fun) jumped out at me as I read it: More than 50 energy company employees have been seconded to the government free of charge, and dozens of them to the department of energy and climate change.

Hmm…. /site:gov.uk (secondment OR seconded) DECC/

Secondments google gov.uk

How about the gov.uk site?

gov uk secondment

(I don’t know what’s going in the fight between GDS and the data.gov.uk folk ito getting central gov #opendata info discoverable on the web, but the http://www.gov.uk domain seems to be winning out, not least because for departments who’re in that empire, that’s where any data that eventually linked to from data.gov.uk will actually be published?)

So – it seems folk have been FOIing this sort of information, but it doesn’t look as if this sort of information is being published according to a regular schedule under an #opendata transparency agenda.

Anyone would thing that the UK government wasn’t in favour of a little bit of light being shone on lobbying activity…

(What did happen to the lobbying bill? Oh, I remember, it got through in a form that doesn’t allow for much useful light shedding at all (enactment), and now Labour want to repeal it.)

I guess I could put a request in to the ODUG (Open Data User Group) for this data to be released as open data, but my hunch is it’s not really the sort of thing they’re interested in (I get the feeling they’re not about open data for transparency, but (perhaps unfairly…?!) see them more as a lobbying group (ODUG membership) for companies who can afford to buy data but who would rather the tax payer pays for its collection and the government then gifts it to them).

More direct would be to find a way of automating FOI requests using something like WhatDoTheyKnow that would fire off an FOI request to each central government department once a month asking for that previous months’ list of secondments into and out of that department in the preceding month (or in the month one or two months preceding that month if they need a monthly salary payment cycle or two for that data to become available).

Of course, it does seem a bit unfair that each government department should have to cover the costs of these requests, but as it stands I can’t make an FOI request of companies that choose to engage in this sort of presumably public service.

Making private companies offering public services under contract subject to FOI does seem to be on the agenda again though, after being knocked back around this time last year?:

An extension to the scope of the FOI Act was proposed a few weeks ago in the Public Bill Committee debate of the morning of Tuesday 18 March 2014 on the Criminal Justice & Courts Bill, columns 186-193:

Dan Jarvis: I beg to move amendment 37, in clause 6, page 6, line 29, at end insert—

‘(1A) The Code of Practice must include a requirement that a person carrying out electronic monitoring who is not a public authority as defined by section 3 of the Freedom of Information Act 2000 shall provide information in respect of the carrying out of electronic monitoring in the same manner as if they were such a public authority.’.

The Chair: With this it will be convenient to discuss amendment 38, in schedule 4, page 73, line 25, at end insert—

‘(1A) Where the Secretary of State enters into a contract with another person under paragraph 1(1), and that person is not a public authority for the purposes of section 3 of the Freedom of Information Act 2000, that person shall be designated by the Secretary of State as a public authority for the purposes of that section in relation to that contract.’.

I remind the Committee that this group is about freedom of information provisions as they apply to aspects of the Bill. Members will have the opportunity to debate the detail of secure colleges later.

Dan Jarvis: I serve notice that, unless sufficient assurances are received, we intend to put the amendments to a vote. [ Interruption. ] Dramatic! I sensed for a moment that there was a higher authority raising a concern about these amendments, but I shall plough on regardless, confident in the knowledge that they are true and right.

Anyone who knows the story of Jajo the rabbit will understand what I am about to say. For those members of the Committee who do not know, Jajo was the pet rabbit successfully registered as a court translator and then booked in for shifts following the Ministry of

Column number: 187
Justice’s outsourcing of language service contracts. Jajo’s short-lived translation career says less about his talent and much more about the importance of ensuring that public contracts delivered by private providers are properly managed.
As was touched on, Ministers now have to manage another fall-out. Two private providers of electronic monitoring overcharged the taxpayer by millions of pounds for tagging offenders who had died or moved abroad, or who were already back in prison. That underlines the case for the amendments.

Both amendments would seek to bring non-public providers of public services contracted out under the Bill within the scope of the Freedom of Information Act. Amendment 37 relates to clause 6 and the code of practice that would be issued by the Secretary of State on the processing of data related to electronic monitoring. It would require anyone carrying out monitoring related to the clauses to comply with FOI requests in the same way as public bodies do. Amendment 38 relates to schedule 4 and the arrangements for contracting out secure colleges, which are detailed in part 2. It would require anyone contracted to provide a secure college to comply with freedom of information in the same way. Both our proposals are worthy of consideration by the Committee.

We all know that the landscape of how public services are delivered is changing. The Government spend £187 billion on goods and services with third parties each year, about half of which is estimated to be on contracting out services. About half of all spending on public services now ends up in private providers’ hands and more and more private providers are bidding to take on the responsibility and financial rewards that come with large-scale public contracts. As outsourcing is stepped up, more and more information about public services and public money is being pulled out of the public domain. That presents a particular challenge that we must tackle.

As the Information Commissioner told the Justice Committee last year,

“if more and more services are delivered by alternative providers who are not public authorities, how do we get accountability?”

The rewards that third parties stand to gain need to go hand in hand with the duties of transparency and information sharing. The public should be able to ask about how, and how well, the service they are paying for is being run.

The Freedom of Information Act does provide for supply-chain companies to be considered to be holding information on behalf of a public authority. In practice, however, contracted providers in the justice sector are not subject to anywhere near the same transparency requirements as publicly-run services. Private prisons, for example, are not subject to FOI in the same way as public prisons and the experience of G4S, Serco and others will have influenced many other companies not to be as forthcoming as they might have been. That is why we need to build freedom of information into the contracts that the Government make with third parties.

The Committee will be aware that such an approach was recommended by the Public Accounts Committee in its excellent report published last week. It made the

Column number: 188
point that many Departments are not providing information on how those contracts work on the grounds of commercial confidentiality. The public will not accept that excuse for much longer.
Let me conclude my remarks by offering the Committee a final quote. Someone once said:

“Information is power. It lets people hold the powerful to account”

and it should be used by them to hold their

“public services to account”.

I agree with the Prime Minister. Two years ago, he spoke about

“the power of transparency”


“why we need more of it.”

He also spoke of leading

“the most transparent Government ever.”

Labour has pledged that the next Labour Government will deal with the issue by bringing companies providing public contracts into the scope of FOI legislation.

Freedom of information can be uncomfortable. It can shed light on difficult issues and be problematic for Government Ministers, but that is the point. The Committee has the opportunity today to improve the Bill and to get a head start.

Dr Huppert: I will not detain the Committee. I share the concern about the lack of FOI for private organisations providing public services. My colleagues and I have expressed concerns about that for many years, and the previous Government were not very good at accepting that. It is good news that the Labour party may undo that error.

Mr Slaughter: Can the hon. Gentleman say what steps he and the coalition have taken to extend FOI in the past four years?

Dr Huppert: Not as many as I would like, but we have seen progress in some areas; we did not see any at all when the hon. Gentleman was a Minister. I hope we will see the correct drive. I share the concern that we need transparency when public services are delivered by private companies. They must not be shielded. I look forward to hearing what the Minister has to say, because he has commented on such issues before.

It is important that the matter should be dealt with on a global scale. I think the shadow Minister would agree that the case is broader. I hope to hear from the Minister that there will be more work to look at how the issue can be addressed more generally, rather than just in a specific case. That would probably require amendment of the Freedom of Information Act. That is probably the best way to resolve the issue, rather than tacking it on to this area, but I absolutely share the concerns. I hope we can see more transparency, both from the Government—we are seeing that—and from the private sector as it performs public functions.

Yasmin Qureshi: The Justice Committee, of which I am a member, looked into the Freedom of Information Act and how it has been operating since it was passed many years ago. We spoke to different groups of people,

Column number: 189
representatives of councils and local authorities, the Information Commissioner and pressure groups. Generally, the view was that the Freedom of Information Act has been a force for good. The thing that people mentioned time and again was the fact that it applies only to public authorities and has a narrow remit in private companies. A lot of concern was expressed about that.
As my hon. Friend the Member for Barnsley Central said, just under £200 billion is being spent by the Government for private companies to carry out public work. The number of outsourcings could increase, especially in the criminal justice system. In the probation service there will be contracting out and privatisation, as well as changes in the criminal justice system in relation to legal aid and suchlike. We have concerns about the criminal justice system and the number of companies that will be carrying out work that the state normally does. It is an important issue.

Will the Minister give us an undertaking for whenever Government money is given to carry out work on behalf of the Government? Local authorities and Government Departments have to provide information, and it should be the same for private companies. At the moment, as the shadow Minister mentioned, the agencies providing some of the public work give some information, but it is not enough.

It is often hard to get information from private companies. It is important for the country that we know where public money is being spent and how private companies respond to such things. We can have party political banter, but freedom of information was introduced many years ago and has been working well. Freedom of information needs to be extended in light of the new circumstances. I ask for a clear commitment from the Government that they will encapsulate that in the Bill. They now have that opportunity; the Labour party has said that, if it was in government, it would certainly do so. The lacunae and the gaps would be addressed by the amendment, which would make it clear exactly how the regime applies. [Interruption.]

10.30 am
The Chair: I apologise for the background noise. We are looking into the cause.

Jeremy Wright: Thank you, Mr Crausby. I hope Jajo the rabbit is not responsible.

As the hon. Member for Barnsley Central said, amendment 37 seeks to introduce a requirement as to the contents of the code of practice that the Secretary of State will issue under proposed new section 62B of the Criminal Justice and Court Services Act 2000, which is to be introduced through clause 6. The Secretary of State would have to include provisions in the code of practice requiring providers of outsourced electronic monitoring services to make information available in the same manner as if they were subject to the provisions of the Freedom of Information Act. The aim of the amendment seems essentially to extend the Act to providers of electronic monitoring not already subject to its provisions.

Amendment 38 has the same basic intention in that it seeks to extend the Freedom of Information Act to providers of secure colleges that have entered a contract with the Secretary of State to do so under schedule 4. The approach differs, however, because amendment 38

Column number: 190
would extend the Act directly, whereas amendment 37 seeks to extend its obligations through code of practice guidance.
In other words, both amendments would require private providers not currently subject to the Freedom of Information Act to make information available both in response to FOI requests and proactively through publication schemes. Section 5 of the Act already provides a power to extend the Act’s provisions to contractors providing public services. For reasons I will try to outline, the Government do not currently propose to adopt that approach and are adopting an alternative method to ensure transparency. I am aware, however, of the long-standing and serious concerns raised on the position under the Act of private providers of public services. It might help the hon. Member for Hammersmith to know that the Government are committed to, and have taken steps to extend, the Act. More than 100 additional organisations have been included since 2010, and we are considering other ways in which its scope may be widened.

The issue of outsourced public services was considered in some detail during post-legislative scrutiny of the Freedom of Information Act by the Select Committee on Justice in 2012. I do not know whether the hon. Member for Bolton South East was a member of the Committee of that time, but the Committee rightly issued a reminder that

“the right to access information is crucial to ensuring accountability and transparency for the spending of taxpayers’ money”.

The Committee recommended the use of contractual provisions, rather than the formal extension of the Act, to ensure that transparency and accountability are maintained. In particular, the Committee said:

“We believe that contracts provide a more practical basis for applying…outsourced services than partial designation of commercial companies under section 5 of the Act”.

The Committee also feels that

“the use of contractual terms to protect the right to access information is currently working relatively well.”

The Government’s approach is consistent with that recommended by the Justice Committee.

In addition to information being made available proactively, the Government are taking steps to issue a revised code of practice under section 45 of the Freedom of Information Act to promote transparency on outsourced public services in response to FOI requests. The code of practice will be issued later this year and will promote and encourage the use and enforcement of contractual obligations to ensure that private bodies not subject to the Act provide appropriate assistance where information about outsourced public services is requested from bodies that are subject to the Act.

The Government recognise that only a small amount of information held by private providers is currently often formally subject to the Act. Our code of practice will encourage public authorities to go further, to interpret their freedom of information obligation broadly and to release more information on a voluntary basis, where it would be in the public interest to do so. In the event of non-compliance, it will also be possible for the Information Commissioner to issue and publish a practice recommendation setting out steps that, in his view, the public authority should take to promote conformity with the guidance.

Column number: 191
Mr Slaughter: I seem to remember taking part in the Westminster Hall debate arising out of the Justice Committee’s deliberation and I do not think that it was very happy with the approach that the Government are taking, particularly where they are seeking to restrict freedom of information further. Does the hon. Gentleman accept on the basis of what he has just said that this will not be a level playing field and that the same requirements that apply to public bodies will not apply to private organisations undertaking an effectively identical role? Does he accept that, whatever the merits of his scheme, it does not to far enough and does not address the comments of my hon. Friend the Member for Barnsley Central?

Jeremy Wright: The hon. Gentleman will recognise that the organisations we are talking about extending the provisions of the Act to cover vary hugely in size and level of resources. The concern is to draw the appropriate balance between giving correct access to information and not imposing intolerable burdens on organisations, particularly smaller ones. That is the balance that has to be struck. We are looking at ways in which we can continue to make public authorities responsible for supplying information but ensure that it comes from the place where it originated, which may be those other organisations.

Mr Slaughter: That is a different argument and one that is often tried. It was tried in relation to universities and to the smaller district councils much beloved of the hon. Member for Bromley and Chislehurst. There are already limitations within the Act. There are safeguards for organisations in terms of the amount of time and cost. Why are they not sufficient?

Jeremy Wright: As I said, there is a balance to be struck. We attempt to strike that balance correctly with our proposals. If I can explain what we want to do a little more fully, perhaps the hon. Gentleman will be reassured—although frankly I doubt it. There is an opportunity for us to look at the issue in a sensible way with the code of practice. Applying our forthcoming code of practice guidance across the public sector will ensure that transparency and response to freedom of information requests will be maintained in a consistent way. This is preferable—I agree with my hon. Friend the Member for Cambridge—to the more piecemeal approach promoted by amendments 37 and 38.

The success of our own code of practice will be monitored by the Ministry of Justice and the Information Commissioner. We were clear in our response to post-legislative scrutiny of the Freedom of Information Act that, should this approach yield insufficient dividends, we will consider what other steps are necessary. In summary, we are committed to ensuring transparency in relation to all outsourced public services, including electronic monitoring and secure colleges. We are taking steps to ensure that through the code of practice to be issued later this year. On that basis, I invite the hon. Gentleman to withdraw his amendment.

Yasmin Qureshi: The Minister referred to the Select Committee on Justice and its recommendations. As you know, without going into the detail of that discussion, Select Committee recommendations sometimes tend to

Column number: 192
be compromises. At the time, three issues were in the mind of the Select Committee. First, it did not realise that a legislative opportunity would come so soon in which to put the measure in a more codified way with a clearer legal obligation. Secondly, there was quite a lot of discussion about private companies.
The Select Committee accepted that the Freedom of Information Act should not apply to purely private companies carrying out purely private work; it was not really arguing against that. However, here we have an opportunity to codify once and for all in legislation the provision that the FOIA should apply whenever public money is paid to a private company to carry out work. That would be a fairly straightforward provision. I do not see why we need to go down the complicated route of using a code of practice, putting in a specific provision in a new contract each time something happens. Why can we not just have a general provision that applies to every situation?

Jeremy Wright: I was a member of the Justice Committee before the hon. Lady was, so I understand her point that recommendations of the Select Committee are a matter of discussion and compromise. However, they are made on a cross-party basis, and paid all the more attention to for that reason. I quoted directly from the Select Committee’s conclusions in what I said earlier.

On the hon. Lady’s other point, this may be an earlier legislative opportunity than the Select Committee anticipated, but of course, it is only an opportunity in relation to specific policies. Again, I rather agree with the point made earlier by my hon. Friend the Member for Cambridge: there is an argument for addressing the issue, not on a piecemeal basis, but more comprehensively.

The hon. Lady’s final point is that the approach that we have set out—using a code of practice—is inadequate and that a statutory approach should be introduced by amending primary legislation. An initial approach of using a code of practice is a sensible one. She will recognise that amendment 37, tabled by the hon. Member for Barnsley Central, deals with a requirement in a code of practice, not primary legislation. Amendment 38 is different, but in relation to electronic monitoring, on which a number of concerns have been expressed, the hon. Gentleman’s chosen vehicle is a code of practice. The code of practice approach appears to be welcomed by both sides of the Committee.

Dan Jarvis: I have listened carefully to the Minister’s response. Clearly, we will want to look carefully at the detail of what he has said about a code of practice.

I agree with my hon. Friend the Member for Bolton South-East that the Committee has an opportunity this morning to make progress on redefining the freedom of information. I have heard the Minister’s response to that point, but the reality is that the move would be popular with the public.

There is no doubt that the landscape in which public services are delivered is changing. The Opposition have pledged to reform freedom of information if we are in government from 2015. I am mindful of the Prime Minister’s comments, which I quoted earlier. He said:

Column number: 193
“Information is power. It lets people hold the powerful to account”,

and it should be used by them to hold their public services to account.

Mike Kane: Does my hon. Friend agree that, as the contracting out of public services expands, the public’s right to information shrinks?

Dan Jarvis: I agree absolutely. There is a degree of inevitability that we will see change in the area. The debate is about how we do it, and it is important that we have that debate. We have tabled the amendments partly so that we can take the opportunity to debate such issues.

Mr Slaughter: There is another point here, which is that the Ministry of Justice is particularly vulnerable on the issue. We have had the privatisation of the probation service and the scandals regarding tagging. We will come to later in the Bill to proposals about the externalisation of the collection of fines and other matters. First, that is going on wholesale in the Department, and secondly, it is defective in many aspects. It is particularly relevant that the Minister should accept that the proposals in the Bill are not sufficient.

Dan Jarvis: My hon. Friend is right. In the context of the delivery of public services within the Ministry of Justice remit, this is a particularly relevant, timely and important issue. It has been incredibly useful to have the opportunity to debate it owing to the tabling of the amendments.

10.45 am
I mentioned that I was mindful of the Prime Minister’s comments, and I am mindful of the fact that the Justice Secretary has also indicated a desire to reform freedom of information. Given that there is a general acknowledgment that the status quo is not acceptable and despite what the Minister has said in response to our amendment, I will press it to a vote.

The amendment was defeated.

An hour or so later, the government took this line:

Daily Hansard, Commons, Tuesday 18 March 2014 – c.639

Freedom of Information Act
23. Lindsay Roy (Glenrothes) (Lab): What plans he has to bring forward legislative proposals to expand the scope of the Freedom of Information Act 2000.

The Minister of State, Ministry of Justice (Simon Hughes): There has been good progress in extending the implementation of the Freedom of Information Act because the coalition Government pledged to extend its scope to provide greater transparency. We extended it in 2010 to academies, in 2011 to the Association of Chief Police Officers, the Financial Ombudsman Service and the Universities and Colleges Admissions Service, and last year to 100 companies wholly owned by more than one public authority. The next item on the agenda is to do with Network Rail, and we are awaiting a view from the Department for Transport as to whether it thinks it would be appropriate for that to be implemented this year.

Lindsay Roy: What benefits have accrued to the Government and citizens from the implementation of the Act, and when does the Minister plan to extend its scope further?

Simon Hughes: We intend to extend it further as soon as is practical. One specific issue that I hope will be of interest to the hon. Gentleman—as it is to colleagues of his, including those who have come to see me about it—is that we intend to publish a revised code of practice to make sure that private companies that carry out public functions have freedom of information requirements in their contracts and go further than that. We hope that that will be in place by the end of this year.

Mr Mark Harper (Forest of Dean) (Con): There is one area where the Minister should perhaps look at narrowing the scope of the Act, because my understanding is that requests can be made by anybody anywhere on the face of the earth; they do not have to be British citizens. It is not the role of the British Government to be a taxpayer-funded research service for anyone on the globe. May I suggest that he narrow the scope to those for whom the Government work—citizens of our country?

Simon Hughes: I well understand my hon. Friend’s point. There will be two consultations this year: first, on precisely such issues about the scope of the current legislation to make sure that it is not abused while we retain freedom of information as a principle of Government; and secondly, on extending it to other areas where we have not gone so far.

Dr Huppert:I read out the quote from someone who has made the position clear when it comes to private companies carrying out public functions. Indeed, the code of practice has exactly the wording used in amendment 11, which the hon. Gentleman supported when we debated it on Tuesday. I do not want to take up too much of the Chairman’s kindness to discuss an issue that was rejected at that point, but it is happening as we wanted.

The matter was also touched upon a couple of days later in a Public Bill Committee on the Criminal Justice and Courts Bill (Official Report, Thursday 20 March 2014, 257-259) where accountability around public contracts delivered by private provides was being discussed:

Mr Slaughter: Absolutely not. I hope that the hon. Gentleman has read the article about Jago the rabbit that my hon. Friend the Member for Barnsley Central (Dan Jarvis) and I wrote for The Independent yesterday [It’s time we extended Freedom of Information to public services run by private companies – just ask Jago the Rabbit], which dealt with what should be done, which is to bring these companies within the ambit of FOI, and what the Minister of State did—with his usual skill, shall we say?—at Justice questions on Tuesday. He implied that that was what was going to happen, whereas in fact he was doing nothing more than putting round the line that the Cabinet Office has already indicated.

If I am wrong about that, I will give way in a moment and the hon. Gentleman can come back to me, but my understanding is that the Government—both parts of it, as long as they are just about coalescing—are of the view that the contracts that are drawn up should include this notional transparency. That is to say that they will encourage public authorities to encourage private companies to put clauses into contracts that will expose as much as possible, within the realms of commercial confidentiality. So the contracts will be open, with publication on websites and so forth of as much information about the contract as the two parties think fit. What we will not have is a duty on those private companies—in so far as they are carrying out public functions—to comply with the terms of the Freedom of Information Act, as would be the case in the public sector.

I accept that they are two sides of the same coin. On the one hand, of course it is a good idea that the information is made available voluntarily, but if it is not—either because the company does not choose to do so or because the contract is not drafted sufficiently well to ensure that it must—the citizen must have the right, through FOI, to require that information to be made available. As far as I am concerned, that is not what was said on Tuesday. I know that there is consultation going on, but if it is the intention of the Government—at least the Liberal Democrat part of the Government—to follow the line taken by my right hon. Friend the Member for Tooting (Sadiq Khan), the shadow Lord Chancellor, which he has repeated often in recent months, and require all those private companies performing public functions to come within the requirements of the Freedom of the Information Act, I would be pleased if the hon. Gentleman said so now.

Mr Slaughter:I take from that comment that even the hon. Gentleman does not understand what the Minister of State, Ministry of Justice, the right hon. Member for Bermondsey and Old Southwark, says, so opaque is it. If nobody, including the Minister, is going to answer my question, the answer will no doubt come out in the wash on a later occasion. However, it seems to me that that is not what is being promised. If it were, the Minister would be jumping up and claiming credit for it, but he is not. Therefore, I assume that that is not the case.

The significance of that is that those four companies about which I have just raised doubts—G4S, Serco, Capita, and we can safely add Atos—all told the Public Accounts Committee that they were prepared to accept the measures that the Committee proposed. It therefore appears that the main barrier to greater transparency lies within Government.

That is where we are. Even the companies that want to put themselves and the interests of their shareholders first are more keen on transparency and on answering the legitimate questions that are being asked by everyone— from ourselves to the chief inspector of prisons—than this Government are.

I say that because if we are to take this further leap down that path, it is only right that the Government do not just challenge, as the Minister has said, acknowledged frauds, but look at the entire performance behaviour, as well as the number of available companies that could step into the breach and deal with these matters.

What we must conclude from the conjunction of clauses 17 and 18 is that, first, the Government are prepared to take this leap in the dark, in terms of the reconfiguration of the youth estate and, secondly, that they are prepared to leave that entirely in the hands of the people who failed so many times in so many contracts, not least in running parts of the adult prison service.

For more on some of the specifics, see the House of Commons Public Accounts Committee report on “Contracting out public services to the private sector”, which for example recommended “that the Cabinet Office should explore how the FOI regime could be extended to cover contracts with private providers, including the scope for an FOI provision to be included in standard contract terms; that neither the Cabinet Office nor departments should routinely use commercial confidentiality as a reason for withholding information about contracts with private providers; [and that] The Cabinet Office should set out a plan for departments to publish routinely standard information on their contracts with private providers”.

There’s also a couple of related private members bills floating around at the moment – Grahame Morris’ empty Freedom of Information (Private Healthcare Companies) Bill 2013-14 “to amend the Freedom of Information Act 2000 to apply to private healthcare companies”, and Caroline Lucas’ Public Services (Ownership and User Involvement) Bill 2013-14 “to put in place mechanisms to increase the accountability, transparency and public control of public services, including those operated by private companies”. The latter >a href=”http://www.publications.parliament.uk/pa/bills/cbill/2013-2014/0160/cbill_2013-20140160_en_2.htm#l1g5″>proposes:

5 Transparency
(1) Where a relevant authority starts the process of procurement for a public services contract, it must make available to the public details of all bids received prior to the conclusion of the procurement exercise.
(2) Where a relevant authority enters into a public services contract, details of that contract shall be made available to the public within 28 days of the procurement decision.

6 Freedom of information
(1) The Secretary of State must designate as a public authority, pursuant to section 5(1)(b) of the Freedom of Information Act 2000, companies or other bodies which enter into a public services contract.
(2) “Public services contract” has the meaning contained within section 8 of this Act.
(3) The Secretary of State shall maintain a list of companies designated under section 6(1) of this Act.
(4) Requests under the Freedom of Information Act 2000 in respect of such companies or bodies can only be made in respect of information relevant to the provision of a public services contract.
(5) The Secretary of State must designate as a public authority, pursuant to section 5(1)(b) of the Freedom of Information Act 2000, any utility company subject to regulation by regulatory authorities as defined in section 8.

Finally, on the accountability and transparency thing, there’s a consultation on at the moment regrading “smaller authorities with an annual turnover not exceeding £25,000, including parish councils, [who] will be exempt from routine external audit” but instead will be subject to a transparency code (Draft transparency code for parish councils – consultation).

Related: Spending & Receipts Transparency as a Consequence of Accepting Public Money? If you accept public money for contracts that would otherwise be provided by a public service you should be subject to the same levels of FOI and transparency reporting. Why should public services have to factor this in to their bids for running a service when private companies don’t?

Other reading to catch up on: Commons Public Administration Select Committee [PASC] Report on Statistics and Open Data (evidence).

Written by Tony Hirst

April 6, 2014 at 10:25 am

Posted in opengov, Policy, Stirring

Tagged with , ,

Demographically Classed

So it seems that in a cost-recovered data release that was probably lawful then but possibly wouldn’t be now* – Hospital records of all NHS patients sold to insurers – the Staple Inn Actuarial Society Critical Illness Definitions and Geographical Variations Working Party (of what, I’m not sure? The Institute and Faculty of Actuaries, perhaps?) got some Hospital Episode Statistics data from the precursor to the HSCIC, blended it with some geodemographic data**, and then came to the conclusion that “that the use of geodemographic profiling could refine Critical illness pricing bases” (source: Extending the Critical Path), presenting the report to the Staple Inn Actuarial Society who also headline branded the PDF version of the report? Maybe?

* House of Commons Health Committee, 25/2/14: 15.59:32 for a few minutes or so; that data release would not be approved now: 16.01:15 reiterated at 16.03:05 and 16.07:05

** or maybe they didn’t? Maybe the data came pre-blended, as @amcunningham suggests in the comments? I’ve added a couple of further questions into my comment reply… – UPDATE: “HES was linked to CACI and Experian data by the Information Centre using full postcode. The working party did not receive any identifiable data.”


“In a story published by the Daily Telegraph today research by the IFoA was represented as “NHS data sold to insurers”. This is not the case. The research referenced in this story considered critical illness in the UK and was presented to members of the Staple Inn Actuarial Society (SIAS) in December 2013 and was made publically available on our website.

“The IFoA is a not for profit professional body. The research paper – Extending the Critical Path – offered actuaries, working in critical illness pricing, information that would help them to ask the right questions of their own data. The aim of providing context in this way is to help improve the accuracy of pricing. Accurate pricing is considered fairer by many consumers and leads to better reserving by insurance companies.

There was also an event on 17 February 2014.

Via a tweet from @SIAScommittee, since deleted for some reason(?), this is clarified further: “SIAS did not produce the research/report.”


The branding that mislead me – I must not be so careless in future…


Many of the current agreements about possible invasions of privacy arising from the planned care.data release relate to the possible reidentification of individuals from their supposedly anonymised or pseudonymised health data (on my to read list: NHS England – Privacy Impact Assessment: care.data) but to my mind the SIAS report presented to the SIAS suggests that we also need to think about consequences of the ways in which aggregated data is analysed and used (for example, in the construction of predictive models). Where aggregate and summarised data is used as the basis of algorithmic decision making, we need to be mindful that sampling errors, as well as other modelling assumptions, may lead to biases in the algorithms that result. Where algorithmic decisions are applied to people placed into statistical sampling “bins” or categories, errors in the assignment of individuals into a particular bin may result in decisions being made against them on an incorrect basis.

Rather than focussing always on the ‘can I personally be identified from the supposedly anonymised or pseudonymised data’, we also need to be mindful of the extent to, and ways in, which:

1) aggregate and summary data is used to produce models about the behaviour of particular groups;
2) individuals are assigned to groups;
3) attributes identified as a result of statistical modelling of groups are assigned to individuals who are (incorrectly) assigned to particular groups, for example on the basis of estimated geodemographic binning.

What worries me is not so much ‘can I be identified from the data’, but ‘are there data attributes about me that bin me in a particular way that statistical models developed around those bins are used to make decisions about me’. (Related to this are notions of algorithmic transparency – though in many cases I think this must surely go hand in hand with ‘binning transparency’!)

That said, for the personal-reidentification-privacy-lobbiests, they may want to pick up on the claim in the SIASIFoA report (page 19) that:

In theory, there should be a one to one correspondence between individual patients and HESID. The HESID is derived using a matching algorithm mainly mapped to NHS number, but not all records contain an NHS number, especially in the early years, so full matching is not possible. In those cases HES use other patient identifiable fields (Date of Birth, Sex, Postcode, etc.) so imperfect matching may mean patients have more than one HESID. According to the NHS IC 83% of records had an NHS number in 2000/01 and this had grown to 97% by 2007/08, so the issue is clearly reducing. Indeed, our data contains 47.5m unique HESIDs which when compared to the English population of around 49m in 1997, and allowing for approximately 1m new lives a year due to births and inwards migration would suggest around 75% of people in England were admitted at least once during the 13 year period for which we have data. Our view is that this proportion seems a little high but we have been unable to verify that this proportion is reasonable against an independent source.

Given two or three data points, if this near 1-1 correspondence exists, you could possibly start guessing at matching HESIDs to individuals, or family units, quite quickly…

To ground the binning idea slightly more, here are the geodemographic bins that the report used. They are taken from two widely used geodemographic segmentation tools, ACORN and MOSAIC.

- ACORN (A Classification of Residential Neighbourhoods) is CACI’s geodemographic segmentation system of the UK population. We have used the 2010 version of ACORN which segments postcodes into 5 Categories, 17 Groups and 57 Types.



- Mosaic UK is Experian’s geodemographic segmentation system of the UK population. We have used the 2009 version of Mosaic UK which segments postcodes into 15 Groups and 67 Household Types.



The ACORN and MOSAIC data sets seem to provide data at the postcode level. I’m not sure how this was then combined with the HES data, but it seems the SIASIFoA folk found a way (p 29) [or as Anne-Marie Cunningham suggests in the comments, maybe it wasn't combined by SIASIFoA - maybe it came that way?]:

The HES data records have been encoded with both an ACORN Type and a Mosaic UK Household Type. This enables hospital admissions to be split by ACORN and Mosaic Type. This covers the “claims” side of an incidence rate calculation. In order to determine the exposure, both CACI and Experian were able to provide us with the population of England, as at 2009 and 2010 respectively, split by gender, age band and profiler.

This then represents another area of concern – the extent to which even pseudonymised data can be combined with other data sets, for example based on geo-demographic data. So for example, how are the datasets actually combined, and what are the possible consequences of such combinations? Does the combination enrich the dataset in such a way that makes it easier for use to deanonymise either of the original datasets (if that is your primary concern); or does the combination occur in such a way that it may introduce systematic biases into models that are then produced by running summary statistics over groupings that are applied over the data, biases that may be unacknowedged (to possibly detrimental effect) when the models are used for predictive modelling, pricing models or as part of policy-making, for example?

Just by the by, I also wonder:

- what data was released lawfully under the old system that wouldn’t be allowed to be released now, and to whom, and for what purpose?
– are the people to whom that data was released allowed to continue using and processing that data?
– if they are allowed to continue using that data, under what conditions and for what purpose?
– if they are not, have they destroyed the data (16.05:44), for example by taking a sledgehammer to the computers the data was held on in the presences of NHS officers, or by whatever other means the state approves of?

See also: Is the UK Government Selling You Off?. For more on data linkage, see Some Random Noticings About Data Linkage.

Written by Tony Hirst

February 25, 2014 at 7:33 pm

Posted in Data, Policy, privacy

Tagged with

Local Council Spending Data – Time Series Charts

In What Role, If Any, Does Spending Data Have to Play in Local Council Budget Consultations? I started wondering about the extent to which local spending transparency data might play a role in supporting consultation around new budgets.

As a first pass, I’ve popped up a quick application up at http://glimmer.rstudio.com/psychemedia/iwspend2013_14/ [if that's broken, try this one] (shiny code here). You can pass form items in via the URL (except to set the Directorate – oops!), and also search using regular expressions, but at the moment still need to hit the Search button to actually run the search. NOTE – there’s a little bug – you need to hit the Search button to get it to show data; note – selecting All directorates and no filter terms can be a bit slow to display anything…


- http://glimmer.rstudio.com/psychemedia/iwspend2013_14/?expensesType=(oil)|(gas)|(electricity)

I’ve started exploring various views over the data, but these need thinking through properly (in particular with respect to finding out views that may actually be useful!)

iw spend music expneses type

Hmm… did the budget change directorate?!

IW spend - music service area

IW spend music suppliers

Some more views over the suppliers tab – I started experimenting with some tabular views in the suppliers tab too…

IW spend music suppliers table 1

IW spend music suppliers table 2

This is all very “shiny” of course, but is it useful? From these early glimpses over the data, can you think of any ways that a look at the spending data might help support budget consultations? What views over the data, in particular, might support such an aim, and what sort of stories might we be able to tell around this sort of data?

Written by Tony Hirst

November 6, 2013 at 11:27 am

Posted in Policy, Rstats

Tagged with ,

What Role, If Any, Does Spending Data Have to Play in Local Council Budget Consultations?

It seems to be that time of year when my local council is consulting about next year’s budgets, running a series of roadshows (which I missed) backed up by an online consultation (but no briefing or discussion documents?)

The online consultation opens with asking for opinions about the extent to which the council should support various services:

Adult Social Care, Asset & Estates Management, Bereavement Services, Central Support Services (Legal, Finance, ICT, Human Resources, Shared Services, Corporate & Democratic Core), Childrens Social Care, Coastal Management, Community Safety, Concessionary Fares, Coroner, Cowes Chain Ferry, Culture & Heritage, Economic Development, Education Psychology/Special Education Needs, Education Welfare Service, Elections & Land Charges, Environmental Health, Licensing, Trading Standards, Fire & Rescue, Harbours, Highways, Housing, Leisure & Recreation, Libraries, Parking Services, PFI Contract, Planning & Building Control, Public Conveniences, Public Health, Registrars, Revenues & Benefits, Schools Infrastructure/buildings, School Music Service, School Transport, Sports & Play Development, Subsidised Buses, Tourism, Waste Management, Workforce Development, Youth Service

The council also publishes a Budget Book which identifies intended savings in three directorate areas (Economy & Environment, Community Wellbeing & Social Care, Schools & Learning).

Budgets are allocated in different service areas according to the Service Reporting Code of Practice for Local Authorities (SeRCOP):

IW revenue budget summary 2013-14

The Revenue Budget Service Analysis breaks things down further within each service area:

Adult social care

IW council adult social care 2013-14

Central Services and Capital Costs

IW council central services and corporate costs 2013-14

Childrens Social Care

IW council childrens social coare 2013-14


Iw council education 2013-14

Cultural and related services

iw cultural and related services

Environmental Services

IW environmental services

Planning and development services

IW planning and development services

Fire and Rescue Services

IW fire and rescue services

Highways and Transport Services

IW highways and transport services

Housing Services

Iw housing services 2013-14

Local spending data

Naive as I am, I started wondering about the extent to which we might be able to map spending items released through the council’s Spending and Finance (Transparency) data releases.

So for example, if we know a service area has a particular annual budget, and we also get to see certain declarations of spend within that are (maybe reconciled to contracts and other procurement information), then we might start to be able to pull together a picture about the range of activities associated with the service area based on the trace shadows those activities throw off in the form of spending items.

If a budget cut is proposed to a service area, and it applies proportionally to the spend, or to particular areas of the spend, we might start to get a feeling about what the practical consequences of the cut might actually be.

So that’s the proposition – but how far can we get when we look at the spending data itself?

Reviewing the spending data to date for the 2013-14 financial year, we see spend associated with the following directorates:

  • Childrens Services
  • Community Wellbeing & Social Care
  • Corporate
  • Economy & Environment
  • Resources
  • Chief Executive, Schools & Learning

It’s not necessarily immediately obvious how these map on to the services listed in the budget book (for example, is Childrens Social Care in Childrens Services or Community Wellbeing & Social Care?

The spending data released by the Isle of Wight Council also has two other relevant columns: the Service Area and the Expenses Type. Unfortunately, the Service Area descriptions seem to be rather ad hoc, and whilst the Expense Area terms may follow some standard vocabulary, it’s not obvious which, or how these terms are associated with particular summary spending areas that appear in the budget book.


As a very quick example of the sort of thing we can start to look for, here is an example of spends associated with the keyword music appearing in the spending Service Area:

iw music spend

The School Music Service is one of the areas the council consultation asked about, so if there is to be a cut to the funding of this service that impacts on spend, we might get a feel for what levels of savings are possible on transparently declared spend in that area:

school Music service

So responsibility for the service changed directorate in July?

Here’s the overall cumulative spend:

school music service2

So the transparently declared spend isn’t that much (the majority of spend – not declared in the spending data – is presumably on salaries?), and what there is declared is mainly on travel. As armchair auditors or budget consultees, I guess this provides us with the question: is just over £10k on staff vehicle mileage over the last 6 months a large amount?

We could possibly also get an estimate of the amount of mileage involved (Isle of Wight council mileage rates). As a lower bound on the mileage, if it was all charged at the highest rate (65p per mile) and all the Staff Vehicle mileage was on miles at that rate, it would come to 10490.62 * 100 / 65.0, or just over 16,100 miles; that is, slightly over 2,500 miles per month, or 100 miles per working day.

But again: is this a large number? When you consider that the School Music Service provides “weekly instrumental and vocal tuition” to “some 3,000 pupils a week in 49 primary, secondary and special schools … from a team of 24 qualified and experienced staff” (via about the School Music Service), it doesn’t seem so much?

(I am mindful that doing such exercises can lead to hatchet jobs around small amounts of money on particular public services, and am quite wary of giving examples as a result… What I do hope to show, though, is how this sort of data investigation does do can encourage you to go looking for other sources of data to help make sense of, and further put into context, the data you do have…)


In terms of adult social care, services provided to “older’ people (presumably, 65 and over) appear to be identified in the budget book headings as one coherent group. So what affect might budget cuts in this area have on spending, based on spending items that might be associated with this area?

If we search for mention of the key term elderly, here’s what we get in accumulated spend to date:

                                           Service.Area     sumTot
                   Elderly Mentally Ill Residential Care 3511701.13
                                  Elderly Frail Homecare 2084136.78
                             Elderly Residential Daycare 1725826.30
                            Elderly Frail Nursing Island 1579067.00
                          Elderly Frail Residential Care 1169120.25
                        Elderly Frail Residential Income 1116792.64
                          Elderly Frail Nursing - Island  669237.95
                     Elderly Mentally Ill Nursing Island  558575.44
                          Elderly Frail Managed Accounts  230445.28
                     Elderly Mental Ill Nursing - Island  216668.02
                           Elderly Mentally Ill Homecare  154026.30
                          Elderly Frail Nursing Mainland   33553.91
                                   Elderly Frail Daycare   23377.70
                   Elderly Mentally Ill Nursing Mainland   11761.68
                   Elderly Mental Ill Nursing - Mainland   10243.80
                   Elderly Mentally Ill Managed Accounts    9257.41
                            Elderly Mentally Ill Daycare    4549.10
                        Elderly Frail Nursing - Mainland    4515.84
                                Elderly Frail Other Care    2711.83
                         Elderly Mentally Ill Other Care    2550.16
                          Elderly Frail Personal Budgets    1900.64
                       Elderly Frail Community Equipment     924.39
                   Elderly Mentally Ill Personal Budgets     625.24
                            Elderly Frail Non-Res Income     171.15
   Elderly Frail - Day/Other care - Client Contributions      36.50
                     Elderly Mentally Ill Nursing Income   -1510.32
                            Elderly Frail Nursing Income   -2371.82

And here are the main suppliers:

                               Supplier.Name    sumTot
                       ISLAND HEALTHCARE LTD 891433.32
                      REDACTED PERSONAL DATA 836757.77
                         SCIO HEALTHCARE LTD 685499.04
                           SOMERSET CARE LTD 616713.96
               LONDON RESIDENTIAL HEALTHCARE 517033.27

We can also get a feel for what the expense types are:

                       Expenses.Type      sumTot
  Charges from Independent Providers 13504438.81
                Regular Respite Care   214573.11
           Crisis Support for Carers    17951.37
                    Fixed Telephones      924.39
                Professional Service      356.04
               Operational Equipment       36.50
                     Client Expenses       11.83
                Client Contributions    -1185.83
                    Provider Refunds  -619211.92

With many challenges facing the provision of homecare more generally, how is funding allocated in service areas containing the keyterm homecare?

                                Service.Area     sumTot
                      Elderly Frail Homecare 2084136.78
                Physical Disability Homecare  395358.15
               Elderly Mentally Ill Homecare  154026.30
                Learning Disability Homecare  123971.95
              Learning Disabilities Homecare  120884.88
                      Mental Health Homecare  117730.46
                           Dementia Homecare   28467.84
                         Homecare Reablement   20892.01
                    Homecare 18-25 year olds   12519.84
   Children Young Adults Disability Homecare    2962.08


As I mentioned, I didn’t get to go to any of the budget consultation roadshows. But I do wonder whether we can make use of spending data to help us get a feel for some of the services that budgets deliver via spend with third parties. For example, can we use such information as a view on to how cuts to budgets might play out in terms of spending cuts associated with the withdrawal or reduction of corresponding services?

In order to relate spend to budget items described in the budget book, I think there is still some way to go though. What we need is another column in the spending data that relates to headings described in the budget book. (Even better might be columns relating to SerCOP codes?)

PS Hmm, I wonder… is this the sort of topic that could make for an interesting School of Data style community based data expedition? And waht role might the Isle of Wight Armchair Auditor or OpenSpending play in such a thing?

Written by Tony Hirst

November 3, 2013 at 12:06 pm

Posted in Policy

Tagged with , ,

Spending & Receipts Transparency as a Consequence of Accepting Public Money?

One of the things I’ve been pondering lately is the asymmetry that exists between the information disclosures that public bodies are obliged to make compared to private ones. My gut feeling is that the public bodies may be placed at a disadvantage by these obligations compared to the private companies, though I guess I need to find some specific examples of this. (Cost may be one; having to release data that can be used by competitors in a procurement exercise may be another; if you have any good examples, please post them in the comments…)

To start with, let’s see how the field is currently set in the area of “transparency” (at least, in a spending sense).

Central government spend over £25k
The obligation on NHS bodies to publish spending data on transactions over £25,000 appears to come via HM Treasury reporting guidance on Transparency – Publication of spend over £25,000 (9th September, 2010), which was released in support of a Prime Ministerial letter of 31st May 2010 to Secretaries of State:

2.1 Scope
2.1.1 This guidance applies to all parts of central government as defined by the Office for National Statistics, including departments, non‐ministerial departments, agencies, NDPBs, Trading Funds and NHS bodies. There are a limited number of exceptions to the requirement to publish. The Intelligence Agencies are completely exempt from this requirement. The following are also not subject to this requirement:
• Financial and non‐financial public corporations
• Parliamentary bodies
• Devolved Administrations
2.1.2 However it is recommended that these bodies adopt this guidance as best practice. Separate guidance is being prepared for local authorities.
2.1.3 Where an organisation comprises both a central government body and a public corporation (e.g. the BBC), this requirement applies to the part of the organisation that is classed as part of central government. The requirement does not apply to that part of the organisation that is a public corporation.

I’m not sure what power obliges these parts of government to conform to this guidance, or the requirement to publish spending data for transactions of £25,000?

A further letter – published on 7 July 2011 – added further obligations across a range of central government departments, as well as on the NHS.

(For information on Transparency in Procurement and Contracting see this supplier factsheet.)

Local authority spend over £500
By contrast, local authorities seem to be obliged to release spending data as a result of the publication of a Code of Recommended Practice for Local Authorities on Data Transparency (29 September 2011) (via) which was “issued by the Secretary of State for Communities and Local Government in exercise of his powers under section 2 of the Local Government, Planning and Land Act 1980 to issue a Code of Recommended Practice (The Code) as to the publication of information by local authorities about the discharge of their functions and other matters which he considers to be related”, where “local authority” means:

- a county council
– a district council
– a parish council which has gross annual income or expenditure (whichever
is the higher) of at least £200,000
– a London borough council
– the Common Council of the City of London in its capacity as a local authority
or police authority
– the Council of the Isles of Scilly
– a National Park authority for a National Park in England
– the Broads Authority 5
– the Greater London Authority so far as it exercises its functions through the
– the London Fire and Emergency Planning Authority
– Transport for London
– the London Development Agency
– a fire and rescue authority (constituted by a scheme under section 2 of the
Fire and Rescue Services Act 2004 or a scheme to which section 4 of that
Act applies, and a metropolitan county fire and rescue authority)
– a police authority, meaning:
(a) a police authority established under section 3 of the Police Act 1996
(b) the Metropolitan Police Authority
– a joint authority established by Part IV of the Local Government Act 1985
(fire and rescue services and transport)
– joint waste authorities, i.e. an authority established for an area in England by
an order under section 207 of the Local Government and Public Involvement
in Health Act 2007
– an economic prosperity board established under section 88 of the Local
Democracy, Economic Development and Construction Act 2009
– a combined authority established under section 103 of that Act
– waste disposal authorities, i.e. an authority established under section 10 of
the Local Government Act 1985
– an Integrated Transport Authority for an integrated transport area in England

The policy area associated with releasing local spending data is this one: Policy – Making local councils more transparent and accountable to local people

So What?

Writing in the Observer (“Open government? Don’t make me laugh”, September 29th, 2013), columnist Nick Cohen wrote:

Public services have always moved from daylight into darkness when private managers take them over. Ever since Labour passed the Freedom of Information Act in 2000, MPs, journalists, bloggers, academics, campaign groups and concerned citizens have been able to examine a prison, say, or medical service up to the moment of privatisation when the possibility of scrutiny vanished.

Sadiq Khan, Labour’s justice spokesman, grasped the need to extend freedom of information to cover the private recipients of public money…

As it is the job of parliament to hold the executive to account, Khan set a test for G4S. He asked for details of its restraint techniques. The company replied that it could not respond to freedom of information requests. The Ministry of Justice would, even though G4S trained the guards and knew what they did while the ministry did not,

The cloak of secrecy may soon be draped over the public sector as well. The Campaign for Freedom of Information is alarmed – to put it mildly – that ministers are talking about making it all too easy for civil servants to refuse to disclose information that the public needs to know – and once had a right to know.

The keenness with which the coalition is protecting commercial interests explains a ministerial manoeuvre that baffled me at the time. When libel reform came before parliament, I, along with everyone else, assumed that the private contractors moving into the NHS, prison service and just about every other service would not be allowed to sue their critics for libel. Under the far-from-liberal existing law, public authorities could not sue because in a democracy voters were free to speak their minds about the providers of public services even if what they said was not in the best possible taste. Indeed, as taxpayers and as the recipients of services, we had a dual justification for saying what we wanted without the threat that crushing financial penalties would bully us into silence.

In the new market-orientated order the coalition was so keen to embrace, any restriction on robust debate would be unfair as well as undemocratic. A failure to allow free speech would mean that businesses and charities could say what they liked about a local authority bidding for a service, but the local authority could not respond in kind for fear of a writ. The Conservatives would not give an inch. In the name of libel reform, they insisted that the freedom to argue in the public square must be restricted and gave private interests an exemption from criticism they denied to public services.

Whatever your feelings about public services and the extent to which private companies could or should be able to deliver them, it seems to me that the different transparency regulations are simply not fair. Public bodies receive public money, so you may argue that it is only right that they should disclose how they spend it. But when the spend with another organisation is so large that it is essentially devolving a significant part of a public body’s budget for spending by a private company, then that private body should be transparent about the way in which it further spends or otherwise allocates the money.

At the moment, transparency in the UK tries to make it possible for use to see who public bodies give money to. If one public body, A, spends with another public body, B, we can also derive information about the receipts that body B has from other public bodies, such as A. If A spends with private company C, we know that A has spent with C but not how C further disburses the funds. If C purchases a service from public body B (which could be a police authority, for example), we don’t know that C spent the public money with B, nor do we know (from the non-existent recipient column of C’s non-existent transparency spending data) that public body B received public money from public body A via private company C.

There is an asymmetry in the way we have sight of public spend when it passes through private hands.

The solution? How about we require private companies that obtain substantial receipts from public bodies (say, over £25k in a single payment) to account for their spend associated with contract in the matter of payments over £500. If the £25k/£500 limits are too onerous (government inflicting red tape on private enterprise, then how about £250k/£10k breakdown. As to the red tape: the public bodies have to dal with it and they are increasingly competing for the same pot of money with the private companies. Come on, chaps, play fair…

The Other Problem – Centralised Receipts

As well as the loss of oversight into how public money is being spent, there is a secondary problem when it comes to tracking the extent to which bodies both public and private receive public money: how do we find how much public money in total a particular private company has received?

On the one hand, we can easily generate reports that show the total amount of public money spent by public body A with private company C by looking at their spending data. (This may be complicated that different companies within a larger group (eg separate recipients C Services Ltd and C Operations Ltd may both be part of C Ltd) all receive separate payments from the body, but as we get more information about beneficial corporate interests we can start to piece together that piece of the jigsaw.)

On the other, to see all public money receipts by a particular company, we need to collate spend data from every public body and then aggregate all the spend with a particular company to get an idea of how much public money it has received, and in what spending areas, from the public sector. (For an early example of this, see Sketching Substantial Council Spending Flows to Serco Using OpenlyLocal Aggregated Spending Data.)

The solution? How about we require private corporate recipients to disclose all public money they have received as part of the deal associated with receiving that public money (a deal that public bodies have to accede to). Again, we may wish to put a threshold on this, even one that has some sort of symmetry associated with it compared to the spending requirements: say, disclosure of £500+ receipts from local government and £25k+ receipts from government departments and other associated bodies.

PS I note a recent Open Letter to the PM from UK civil society organisations that makes a related request:


by opening up public sector contracts and extending transparency standards and legislation. Endorse and implement a system of ‘Open Contracting’, ensuring public disclosure and monitoring of contracting from procurement to the close of projects, and amend the Freedom of Information Act so that all information held by a contractor in connection with a public service contract is brought within its scope.

Ah, yes… Information asymmetry around FOI requests. Do you know of any companies who operate parking meters on behalf of a local council? I’d like to see if I could get this sort of data from them…

PPS see also the House of Commons Public Administration Committee, who took evidence on Statistics and open data on October 8th, 2013.

Written by Tony Hirst

October 9, 2013 at 9:38 am

Posted in Policy

Tagged with

Some Random Noticings About Data Linkage

Via a post on my colleague, and info law watchdog, Ray Corrigan’s blog – Alas medical confidentiality in the UK, we knew it well… – I note he has some concerns about the way in which the NHS data linkage service may be able to up its game as a result of the creation of the HSCIC – the Health and Social Care Information Centre – and it’s increasing access to data (including personal medical records?) held by GPs via the General Practice Extraction Service (GPES). (The HSCIC itself was established via legislation: Part 9 Chapter 2 of the Health and Social Care Act 2012. As I commented in The Business of Open Public Data Rolls On…, I think we need to keep a careful eye on (proposed) legislation that allows for “information of a prescribed description” to be made available to a “prescribed person” or “a person falling within a prescribed category”, where those prescriptions are left to the whim of the Minister responsible.) (Also via Ray, medConfidential has an interesting review of the HSCIC/GPES story so far.)

Something I hadn’t spotted before was the price list for data extraction and linkage services – just as interesting as the prices are the categories of service:

HSCIC datalinkage services

Here are the actual prices:

HSCIC data linkage price list

Complexity based on time to process-

3. A request is classed as ‘simple’ if specification, production and checking are expected to take less than 5 hours.
4. A request is classed as ‘medium’ if specification, production and checking are expected to take less than 7 hours but more than 5.
5. A request is classed as ‘complex’ if specification, production and checking are expected to take less than 12 hours but more than 7.

Doing a little search around the notion of “data linkage”, I stumbled across what looks to be quite a major data linkage initiative going on in Scotland – the Scotland Wide data linkage framework. There seems to have been a significant consultation exercise in 2012 prior to the establishment of this framework earlier this year: Data Linkage Framework Consultation [closed] [see for example the Consultation paper on “Aims and Guiding Principles” or the Technical Consultation on the Design of the Data Sharing and Linking Service [closed]]. Perhaps mindful of the fact that there may have been and may yet be public concerns around the notion of data linkage, an engagement exercise and report on Public Acceptability of Cross-Sectoral Data Linkage was also carried out (August 2012). A further round of engagement looks set so occur during November 2013.

I’m not sure what the current state of the framework, or its implementation, is (maybe this FOI request on Members and minutes of meetings of Data Linkage Framework Operations Group would give some pointers?) but one component of it at at least looks to be the Electronic Data Research and Innovation Service (eDRIS), a “one-stop shop for health informatics research”, apparently… Here’s the elevator pitch:

edris elevator pitch

Some examples of collaborative work are also provided:

- Linking data from NHS24 and Scottish Ambulance Service with emergency admissions and deaths data to understand unscheduled care patient pathways.
– Working with NHS Lothian to provide linked health data to support EuroHOPE – European Healthcare Outcomes, Performance and Efficiency Project Epidemiology, disease burden and outcomes of diverticular disease in Scotland
– Infant feeding in Scotland: Exploring the factors that influence infant feeding choices (within Glasgow) and the potential health and economic benefits of breastfeeding on child health

This got me wondering about what sorts of data linkage project things like HSCIC or the MoJ data lab (as reviewed here) might get up to. Several examples seem to to provided by the ESRC Administrative Data Liaison Service (ADLS): Summary of administrative data linkage. (For more information about the ADLS, see the Administrative Data Taskforce report Improving Access for Research and Policy.)

The ADLS itself was created as part of a three phase funding programme by the ESRC, which is currently calling for second phase bids for Business and Local Government Data Research Centres. I wonder if offering data linkage services will play a role in their remit? If so, I wonder if they will offer services along the lines of the ADLS Trusted Third Party Service (TTPS), which “provides researchers and data holding organisations a mechanism to enable the combining and enhancing of data for research to which may not have otherwise been possible because of data privacy and security concerns”? Apparently,

The [ADLS TTPS] facility is housed within a secure room within the Centre for Census and Survey Research (CCSR) at the University of Manchester, and has been audited by the Office for National Statistics. The room is only used to carry out disclosure risk assessment work and other work that requires access to identifiable data.”

Another example of a secure environment for data analysis is provided by the the HMRC Datalab. One thing I notice about that facility is that they don’t appear to allow expect researchers to use R (the software list identifies STATA 9/10/11, SAS 9.3, Microsoft Excel, Microsoft Word, SPSS Clementine 8.1/9.0/10.1/11.1/12)?

Why’s this important? Because little L, little D, linked data can provide a much richer picture than distinct data sets…

PS see also mosaic theory

PPS reminded by @wilm, here’s a “nice” example of data linkage from the New York Times… N.S.A. Gathers Data on Social Connections of U.S. Citizens.

PPPS and from the midata Innovation Lab, I notice this claim:

On the 4th of July 2013 we opened the midata Innovation Lab (mIL), on what we call “UK Consumer Independence Day”. So what is it? It’s the UK Government, leading UK companies and authoritative bodies collaborating on data services innovation and consumer protection for a data-driven future. We’ve put together the world’s fastest-built data innovation lab, creating the world’s most interesting and varied datasets, for the UK’s best brands and developers to work with.

The mIL is an accelerator for business to use a rich dataset to create new services for consumers. Designed in conjunction with innovative “Founding Partner” businesses, it also has oversight from authoritative bodies so we can create the world’s best consumer protection in the emerging personal data ecosystem.

The unique value of the lab is its ability to offer a unique dataset and consumer insight that it would be difficult for any one organization to collate. With expert input from authoritative consumer protection bodies, we can test and learn how to both empower and protect consumers in the emerging personal data ecosystem.

And this: “The personal data that we have asked for is focused on a few key areas: personal information including vehicle and property data, transactional banking and credit records, mobile, phone, broadband and TV billing information and utility bills.” It seems that data was collected from 50 individuals to start with.

Written by Tony Hirst

September 28, 2013 at 6:01 pm

Posted in Data, Policy

Opening Research Data on Your Behalf…

I note a pre-announced intention from the Justice Data Lab that they will publish “[t]ailored reports pertaining to the re-offending outcomes of services or interventions delivered by organisations who have requested information through the Justice Data Lab. Each report will be an Official Statistic.

If you haven’t been keeping up, the Justice data lab is a currently free, one year pilot scheme (started April 2013) in which “a small team from Analytical Services within the Ministry of Justice (the Justice Data Lab team) will support organisations that provide offender services by allowing them easy access to aggregate re-offending data specific to the group of people they have worked with” [User Journey].

Here’s how the user journey doc describes the process…

data lab process

…and the methodology:

datalab methodology

which is also described in the pre-announcement doc as follows:

Participating organisations supply the Justice Data Lab with details of the offenders who they have worked with, and information about the services they have provided. As standard the Justice Data Lab will supply aggregate one year proven re-offending rates for that group, and that of a matched control group of similar offenders. The re-offending rates for the organisation’s group and the matched control group will be compared using statistical testing to assess the impact of the organisation’s work on reducing re-offending. The results will then be returned to the organisation in a clear and easy to understand format, with explanations of the key metrics, and any caveats and limitations necessary for interpretation of the results.

The pre-announcement suggests that participating organisations will not only receive a copy of the report, but so will the public… The rationale:

The Justice Data Lab pilot is free at the point of service, paid for through the Ministry of Justice budget. The Ministry of Justice therefore has a duty to act transparently and openly about the outcomes of this initiative. It is anticipated that by making this information available in the public domain, organisations that work with offenders will have a greater evidence base about what works to rehabilitate offenders, and ultimately cut crime.

(Nice to see the MoJ believes in transparency. Shame that doesn’t go as far as timely spending data transparency, but I guess we can’t have it all…)

I think it’s worth taking notice of this pre-announcement for few reasons:

- are such data release mechanisms the result of lobbying pressure? Other government departments have datalabs, such as the HMRC datalab. HMRC recently ran a consultation on the release of VAT registration information as opendata, although concerns have been raised that this may just be a shortcut way of releasing company VAT registration data to credit rating agencies and their ilk…?, so it seems as if they are looking at what data they may be able to open up, and how, maybe in response to lobbying requests from corporate players who don’t want to have to (pay to) collect the data themselves…? Who might have lobbied the MoJ for the results of MoJ datalab requests to be opened up as public data, I wonder?

- are the results gameable, or might they be used as a tool to “attack” a group that is the basis of a research request? For example, can third parties request that the MoJ datalab runs an analysis on the effectiveness of a programme carried out by another party, such as, I dunno, G4S?

- the ESRC is in the process of a multi-stage funding round that will establish a range of research data centres. The first round, to establish a series of Administrative Data Research Centres has now closed (who won?!) and the second – for Business and Local Government Data Research Centres – is currently open. (Phase three will focus on “Third Sector data and social media data”…wtf?!) To what extent might any of the funded research data centres require that summaries of analyses run using datasets they control access to are released as public open data?

Just by the by, I note here the RCUK Common Principles on Data Policy:

Publicly funded research data are a public good, produced in the public interest, which should be made openly available with as few restrictions as possible in a timely and responsible manner that does not harm intellectual property.

Institutional and project specific data management policies and plans should be in accordance with relevant standards and community best practice. Data with acknowledged long-term value should be preserved and remain accessible and usable for future research.

To enable research data to be discoverable and effectively re-used by others, sufficient metadata should be recorded and made openly available to enable other researchers to understand the research and re-use potential of the data. Published results should always include information on how to access the supporting data.

RCUK recognises that there are legal, ethical and commercial constraints on release of research data. To ensure that the research process is not damaged by inappropriate release of data, research organisation policies and practices should ensure that these are considered at all stages in the research process.

To ensure that research teams get appropriate recognition for the effort involved in collecting and analysing data, those who undertake Research Council funded work may be entitled to a limited period of privileged use of the data they have collected to enable them to publish the results of their research. The length of this period varies by research discipline and, where appropriate, is discussed further in the published policies of individual Research Councils.

In order to recognise the intellectual contributions of researchers who generate, preserve and share key research datasets, all users of research data should acknowledge the sources of their data and abide by the terms and conditions under which they are accessed.

It is appropriate to use public funds to support the management and sharing of publicly-funded research data. To maximise the research benefit which can be gained from limited budgets, the mechanisms for these activities should be both efficient and cost-effective in the use of public funds.

See also: Some Sketchnotes on a Few of My Concerns About #opendata

Written by Tony Hirst

September 23, 2013 at 3:41 pm

Posted in Anything you want, Policy

Tagged with ,


Get every new post delivered to your Inbox.

Join 794 other followers