OUseful.Info, the blog…

Trying to find useful things to do with emerging technologies in open education

Archive for the ‘opengov’ Category

Regularly Scheduled FOI Requests as a None Too Subtle Regular OpenData Release Request? And Some Notes on Extending FOI

with one comment

A piece of contextualisation in an interview piece with Green MP Caroline Lucas in Saturday’s Guardian (I didn’t do this because I thought it was fun) jumped out at me as I read it: More than 50 energy company employees have been seconded to the government free of charge, and dozens of them to the department of energy and climate change.

Hmm…. /site:gov.uk (secondment OR seconded) DECC/

Secondments google gov.uk

How about the gov.uk site?

gov uk secondment

(I don’t know what’s going in the fight between GDS and the data.gov.uk folk ito getting central gov #opendata info discoverable on the web, but the http://www.gov.uk domain seems to be winning out, not least because for departments who’re in that empire, that’s where any data that eventually linked to from data.gov.uk will actually be published?)

So – it seems folk have been FOIing this sort of information, but it doesn’t look as if this sort of information is being published according to a regular schedule under an #opendata transparency agenda.

Anyone would thing that the UK government wasn’t in favour of a little bit of light being shone on lobbying activity…

(What did happen to the lobbying bill? Oh, I remember, it got through in a form that doesn’t allow for much useful light shedding at all (enactment), and now Labour want to repeal it.)

I guess I could put a request in to the ODUG (Open Data User Group) for this data to be released as open data, but my hunch is it’s not really the sort of thing they’re interested in (I get the feeling they’re not about open data for transparency, but (perhaps unfairly…?!) see them more as a lobbying group (ODUG membership) for companies who can afford to buy data but who would rather the tax payer pays for its collection and the government then gifts it to them).

More direct would be to find a way of automating FOI requests using something like WhatDoTheyKnow that would fire off an FOI request to each central government department once a month asking for that previous months’ list of secondments into and out of that department in the preceding month (or in the month one or two months preceding that month if they need a monthly salary payment cycle or two for that data to become available).

Of course, it does seem a bit unfair that each government department should have to cover the costs of these requests, but as it stands I can’t make an FOI request of companies that choose to engage in this sort of presumably public service.

Making private companies offering public services under contract subject to FOI does seem to be on the agenda again though, after being knocked back around this time last year?:

An extension to the scope of the FOI Act was proposed a few weeks ago in the Public Bill Committee debate of the morning of Tuesday 18 March 2014 on the Criminal Justice & Courts Bill, columns 186-193:

Dan Jarvis: I beg to move amendment 37, in clause 6, page 6, line 29, at end insert—

‘(1A) The Code of Practice must include a requirement that a person carrying out electronic monitoring who is not a public authority as defined by section 3 of the Freedom of Information Act 2000 shall provide information in respect of the carrying out of electronic monitoring in the same manner as if they were such a public authority.’.

The Chair: With this it will be convenient to discuss amendment 38, in schedule 4, page 73, line 25, at end insert—

‘(1A) Where the Secretary of State enters into a contract with another person under paragraph 1(1), and that person is not a public authority for the purposes of section 3 of the Freedom of Information Act 2000, that person shall be designated by the Secretary of State as a public authority for the purposes of that section in relation to that contract.’.

I remind the Committee that this group is about freedom of information provisions as they apply to aspects of the Bill. Members will have the opportunity to debate the detail of secure colleges later.

Dan Jarvis: I serve notice that, unless sufficient assurances are received, we intend to put the amendments to a vote. [ Interruption. ] Dramatic! I sensed for a moment that there was a higher authority raising a concern about these amendments, but I shall plough on regardless, confident in the knowledge that they are true and right.

Anyone who knows the story of Jajo the rabbit will understand what I am about to say. For those members of the Committee who do not know, Jajo was the pet rabbit successfully registered as a court translator and then booked in for shifts following the Ministry of

Column number: 187
Justice’s outsourcing of language service contracts. Jajo’s short-lived translation career says less about his talent and much more about the importance of ensuring that public contracts delivered by private providers are properly managed.
As was touched on, Ministers now have to manage another fall-out. Two private providers of electronic monitoring overcharged the taxpayer by millions of pounds for tagging offenders who had died or moved abroad, or who were already back in prison. That underlines the case for the amendments.

Both amendments would seek to bring non-public providers of public services contracted out under the Bill within the scope of the Freedom of Information Act. Amendment 37 relates to clause 6 and the code of practice that would be issued by the Secretary of State on the processing of data related to electronic monitoring. It would require anyone carrying out monitoring related to the clauses to comply with FOI requests in the same way as public bodies do. Amendment 38 relates to schedule 4 and the arrangements for contracting out secure colleges, which are detailed in part 2. It would require anyone contracted to provide a secure college to comply with freedom of information in the same way. Both our proposals are worthy of consideration by the Committee.

We all know that the landscape of how public services are delivered is changing. The Government spend £187 billion on goods and services with third parties each year, about half of which is estimated to be on contracting out services. About half of all spending on public services now ends up in private providers’ hands and more and more private providers are bidding to take on the responsibility and financial rewards that come with large-scale public contracts. As outsourcing is stepped up, more and more information about public services and public money is being pulled out of the public domain. That presents a particular challenge that we must tackle.

As the Information Commissioner told the Justice Committee last year,

“if more and more services are delivered by alternative providers who are not public authorities, how do we get accountability?”

The rewards that third parties stand to gain need to go hand in hand with the duties of transparency and information sharing. The public should be able to ask about how, and how well, the service they are paying for is being run.

The Freedom of Information Act does provide for supply-chain companies to be considered to be holding information on behalf of a public authority. In practice, however, contracted providers in the justice sector are not subject to anywhere near the same transparency requirements as publicly-run services. Private prisons, for example, are not subject to FOI in the same way as public prisons and the experience of G4S, Serco and others will have influenced many other companies not to be as forthcoming as they might have been. That is why we need to build freedom of information into the contracts that the Government make with third parties.

The Committee will be aware that such an approach was recommended by the Public Accounts Committee in its excellent report published last week. It made the

Column number: 188
point that many Departments are not providing information on how those contracts work on the grounds of commercial confidentiality. The public will not accept that excuse for much longer.
Let me conclude my remarks by offering the Committee a final quote. Someone once said:

“Information is power. It lets people hold the powerful to account”

and it should be used by them to hold their

“public services to account”.

I agree with the Prime Minister. Two years ago, he spoke about

“the power of transparency”

and

“why we need more of it.”

He also spoke of leading

“the most transparent Government ever.”

Labour has pledged that the next Labour Government will deal with the issue by bringing companies providing public contracts into the scope of FOI legislation.

Freedom of information can be uncomfortable. It can shed light on difficult issues and be problematic for Government Ministers, but that is the point. The Committee has the opportunity today to improve the Bill and to get a head start.

Dr Huppert: I will not detain the Committee. I share the concern about the lack of FOI for private organisations providing public services. My colleagues and I have expressed concerns about that for many years, and the previous Government were not very good at accepting that. It is good news that the Labour party may undo that error.

Mr Slaughter: Can the hon. Gentleman say what steps he and the coalition have taken to extend FOI in the past four years?

Dr Huppert: Not as many as I would like, but we have seen progress in some areas; we did not see any at all when the hon. Gentleman was a Minister. I hope we will see the correct drive. I share the concern that we need transparency when public services are delivered by private companies. They must not be shielded. I look forward to hearing what the Minister has to say, because he has commented on such issues before.

It is important that the matter should be dealt with on a global scale. I think the shadow Minister would agree that the case is broader. I hope to hear from the Minister that there will be more work to look at how the issue can be addressed more generally, rather than just in a specific case. That would probably require amendment of the Freedom of Information Act. That is probably the best way to resolve the issue, rather than tacking it on to this area, but I absolutely share the concerns. I hope we can see more transparency, both from the Government—we are seeing that—and from the private sector as it performs public functions.

Yasmin Qureshi: The Justice Committee, of which I am a member, looked into the Freedom of Information Act and how it has been operating since it was passed many years ago. We spoke to different groups of people,

Column number: 189
representatives of councils and local authorities, the Information Commissioner and pressure groups. Generally, the view was that the Freedom of Information Act has been a force for good. The thing that people mentioned time and again was the fact that it applies only to public authorities and has a narrow remit in private companies. A lot of concern was expressed about that.
As my hon. Friend the Member for Barnsley Central said, just under £200 billion is being spent by the Government for private companies to carry out public work. The number of outsourcings could increase, especially in the criminal justice system. In the probation service there will be contracting out and privatisation, as well as changes in the criminal justice system in relation to legal aid and suchlike. We have concerns about the criminal justice system and the number of companies that will be carrying out work that the state normally does. It is an important issue.

Will the Minister give us an undertaking for whenever Government money is given to carry out work on behalf of the Government? Local authorities and Government Departments have to provide information, and it should be the same for private companies. At the moment, as the shadow Minister mentioned, the agencies providing some of the public work give some information, but it is not enough.

It is often hard to get information from private companies. It is important for the country that we know where public money is being spent and how private companies respond to such things. We can have party political banter, but freedom of information was introduced many years ago and has been working well. Freedom of information needs to be extended in light of the new circumstances. I ask for a clear commitment from the Government that they will encapsulate that in the Bill. They now have that opportunity; the Labour party has said that, if it was in government, it would certainly do so. The lacunae and the gaps would be addressed by the amendment, which would make it clear exactly how the regime applies. [Interruption.]

10.30 am
The Chair: I apologise for the background noise. We are looking into the cause.

Jeremy Wright: Thank you, Mr Crausby. I hope Jajo the rabbit is not responsible.

As the hon. Member for Barnsley Central said, amendment 37 seeks to introduce a requirement as to the contents of the code of practice that the Secretary of State will issue under proposed new section 62B of the Criminal Justice and Court Services Act 2000, which is to be introduced through clause 6. The Secretary of State would have to include provisions in the code of practice requiring providers of outsourced electronic monitoring services to make information available in the same manner as if they were subject to the provisions of the Freedom of Information Act. The aim of the amendment seems essentially to extend the Act to providers of electronic monitoring not already subject to its provisions.

Amendment 38 has the same basic intention in that it seeks to extend the Freedom of Information Act to providers of secure colleges that have entered a contract with the Secretary of State to do so under schedule 4. The approach differs, however, because amendment 38

Column number: 190
would extend the Act directly, whereas amendment 37 seeks to extend its obligations through code of practice guidance.
In other words, both amendments would require private providers not currently subject to the Freedom of Information Act to make information available both in response to FOI requests and proactively through publication schemes. Section 5 of the Act already provides a power to extend the Act’s provisions to contractors providing public services. For reasons I will try to outline, the Government do not currently propose to adopt that approach and are adopting an alternative method to ensure transparency. I am aware, however, of the long-standing and serious concerns raised on the position under the Act of private providers of public services. It might help the hon. Member for Hammersmith to know that the Government are committed to, and have taken steps to extend, the Act. More than 100 additional organisations have been included since 2010, and we are considering other ways in which its scope may be widened.

The issue of outsourced public services was considered in some detail during post-legislative scrutiny of the Freedom of Information Act by the Select Committee on Justice in 2012. I do not know whether the hon. Member for Bolton South East was a member of the Committee of that time, but the Committee rightly issued a reminder that

“the right to access information is crucial to ensuring accountability and transparency for the spending of taxpayers’ money”.

The Committee recommended the use of contractual provisions, rather than the formal extension of the Act, to ensure that transparency and accountability are maintained. In particular, the Committee said:

“We believe that contracts provide a more practical basis for applying…outsourced services than partial designation of commercial companies under section 5 of the Act”.

The Committee also feels that

“the use of contractual terms to protect the right to access information is currently working relatively well.”

The Government’s approach is consistent with that recommended by the Justice Committee.

In addition to information being made available proactively, the Government are taking steps to issue a revised code of practice under section 45 of the Freedom of Information Act to promote transparency on outsourced public services in response to FOI requests. The code of practice will be issued later this year and will promote and encourage the use and enforcement of contractual obligations to ensure that private bodies not subject to the Act provide appropriate assistance where information about outsourced public services is requested from bodies that are subject to the Act.

The Government recognise that only a small amount of information held by private providers is currently often formally subject to the Act. Our code of practice will encourage public authorities to go further, to interpret their freedom of information obligation broadly and to release more information on a voluntary basis, where it would be in the public interest to do so. In the event of non-compliance, it will also be possible for the Information Commissioner to issue and publish a practice recommendation setting out steps that, in his view, the public authority should take to promote conformity with the guidance.

Column number: 191
Mr Slaughter: I seem to remember taking part in the Westminster Hall debate arising out of the Justice Committee’s deliberation and I do not think that it was very happy with the approach that the Government are taking, particularly where they are seeking to restrict freedom of information further. Does the hon. Gentleman accept on the basis of what he has just said that this will not be a level playing field and that the same requirements that apply to public bodies will not apply to private organisations undertaking an effectively identical role? Does he accept that, whatever the merits of his scheme, it does not to far enough and does not address the comments of my hon. Friend the Member for Barnsley Central?

Jeremy Wright: The hon. Gentleman will recognise that the organisations we are talking about extending the provisions of the Act to cover vary hugely in size and level of resources. The concern is to draw the appropriate balance between giving correct access to information and not imposing intolerable burdens on organisations, particularly smaller ones. That is the balance that has to be struck. We are looking at ways in which we can continue to make public authorities responsible for supplying information but ensure that it comes from the place where it originated, which may be those other organisations.

Mr Slaughter: That is a different argument and one that is often tried. It was tried in relation to universities and to the smaller district councils much beloved of the hon. Member for Bromley and Chislehurst. There are already limitations within the Act. There are safeguards for organisations in terms of the amount of time and cost. Why are they not sufficient?

Jeremy Wright: As I said, there is a balance to be struck. We attempt to strike that balance correctly with our proposals. If I can explain what we want to do a little more fully, perhaps the hon. Gentleman will be reassured—although frankly I doubt it. There is an opportunity for us to look at the issue in a sensible way with the code of practice. Applying our forthcoming code of practice guidance across the public sector will ensure that transparency and response to freedom of information requests will be maintained in a consistent way. This is preferable—I agree with my hon. Friend the Member for Cambridge—to the more piecemeal approach promoted by amendments 37 and 38.

The success of our own code of practice will be monitored by the Ministry of Justice and the Information Commissioner. We were clear in our response to post-legislative scrutiny of the Freedom of Information Act that, should this approach yield insufficient dividends, we will consider what other steps are necessary. In summary, we are committed to ensuring transparency in relation to all outsourced public services, including electronic monitoring and secure colleges. We are taking steps to ensure that through the code of practice to be issued later this year. On that basis, I invite the hon. Gentleman to withdraw his amendment.

Yasmin Qureshi: The Minister referred to the Select Committee on Justice and its recommendations. As you know, without going into the detail of that discussion, Select Committee recommendations sometimes tend to

Column number: 192
be compromises. At the time, three issues were in the mind of the Select Committee. First, it did not realise that a legislative opportunity would come so soon in which to put the measure in a more codified way with a clearer legal obligation. Secondly, there was quite a lot of discussion about private companies.
The Select Committee accepted that the Freedom of Information Act should not apply to purely private companies carrying out purely private work; it was not really arguing against that. However, here we have an opportunity to codify once and for all in legislation the provision that the FOIA should apply whenever public money is paid to a private company to carry out work. That would be a fairly straightforward provision. I do not see why we need to go down the complicated route of using a code of practice, putting in a specific provision in a new contract each time something happens. Why can we not just have a general provision that applies to every situation?

Jeremy Wright: I was a member of the Justice Committee before the hon. Lady was, so I understand her point that recommendations of the Select Committee are a matter of discussion and compromise. However, they are made on a cross-party basis, and paid all the more attention to for that reason. I quoted directly from the Select Committee’s conclusions in what I said earlier.

On the hon. Lady’s other point, this may be an earlier legislative opportunity than the Select Committee anticipated, but of course, it is only an opportunity in relation to specific policies. Again, I rather agree with the point made earlier by my hon. Friend the Member for Cambridge: there is an argument for addressing the issue, not on a piecemeal basis, but more comprehensively.

The hon. Lady’s final point is that the approach that we have set out—using a code of practice—is inadequate and that a statutory approach should be introduced by amending primary legislation. An initial approach of using a code of practice is a sensible one. She will recognise that amendment 37, tabled by the hon. Member for Barnsley Central, deals with a requirement in a code of practice, not primary legislation. Amendment 38 is different, but in relation to electronic monitoring, on which a number of concerns have been expressed, the hon. Gentleman’s chosen vehicle is a code of practice. The code of practice approach appears to be welcomed by both sides of the Committee.

Dan Jarvis: I have listened carefully to the Minister’s response. Clearly, we will want to look carefully at the detail of what he has said about a code of practice.

I agree with my hon. Friend the Member for Bolton South-East that the Committee has an opportunity this morning to make progress on redefining the freedom of information. I have heard the Minister’s response to that point, but the reality is that the move would be popular with the public.

There is no doubt that the landscape in which public services are delivered is changing. The Opposition have pledged to reform freedom of information if we are in government from 2015. I am mindful of the Prime Minister’s comments, which I quoted earlier. He said:

Column number: 193
“Information is power. It lets people hold the powerful to account”,

and it should be used by them to hold their public services to account.

Mike Kane: Does my hon. Friend agree that, as the contracting out of public services expands, the public’s right to information shrinks?

Dan Jarvis: I agree absolutely. There is a degree of inevitability that we will see change in the area. The debate is about how we do it, and it is important that we have that debate. We have tabled the amendments partly so that we can take the opportunity to debate such issues.

Mr Slaughter: There is another point here, which is that the Ministry of Justice is particularly vulnerable on the issue. We have had the privatisation of the probation service and the scandals regarding tagging. We will come to later in the Bill to proposals about the externalisation of the collection of fines and other matters. First, that is going on wholesale in the Department, and secondly, it is defective in many aspects. It is particularly relevant that the Minister should accept that the proposals in the Bill are not sufficient.

Dan Jarvis: My hon. Friend is right. In the context of the delivery of public services within the Ministry of Justice remit, this is a particularly relevant, timely and important issue. It has been incredibly useful to have the opportunity to debate it owing to the tabling of the amendments.

10.45 am
I mentioned that I was mindful of the Prime Minister’s comments, and I am mindful of the fact that the Justice Secretary has also indicated a desire to reform freedom of information. Given that there is a general acknowledgment that the status quo is not acceptable and despite what the Minister has said in response to our amendment, I will press it to a vote.

The amendment was defeated.

An hour or so later, the government took this line:

Daily Hansard, Commons, Tuesday 18 March 2014 – c.639

Freedom of Information Act
23. Lindsay Roy (Glenrothes) (Lab): What plans he has to bring forward legislative proposals to expand the scope of the Freedom of Information Act 2000.

The Minister of State, Ministry of Justice (Simon Hughes): There has been good progress in extending the implementation of the Freedom of Information Act because the coalition Government pledged to extend its scope to provide greater transparency. We extended it in 2010 to academies, in 2011 to the Association of Chief Police Officers, the Financial Ombudsman Service and the Universities and Colleges Admissions Service, and last year to 100 companies wholly owned by more than one public authority. The next item on the agenda is to do with Network Rail, and we are awaiting a view from the Department for Transport as to whether it thinks it would be appropriate for that to be implemented this year.

Lindsay Roy: What benefits have accrued to the Government and citizens from the implementation of the Act, and when does the Minister plan to extend its scope further?

Simon Hughes: We intend to extend it further as soon as is practical. One specific issue that I hope will be of interest to the hon. Gentleman—as it is to colleagues of his, including those who have come to see me about it—is that we intend to publish a revised code of practice to make sure that private companies that carry out public functions have freedom of information requirements in their contracts and go further than that. We hope that that will be in place by the end of this year.

Mr Mark Harper (Forest of Dean) (Con): There is one area where the Minister should perhaps look at narrowing the scope of the Act, because my understanding is that requests can be made by anybody anywhere on the face of the earth; they do not have to be British citizens. It is not the role of the British Government to be a taxpayer-funded research service for anyone on the globe. May I suggest that he narrow the scope to those for whom the Government work—citizens of our country?

Simon Hughes: I well understand my hon. Friend’s point. There will be two consultations this year: first, on precisely such issues about the scope of the current legislation to make sure that it is not abused while we retain freedom of information as a principle of Government; and secondly, on extending it to other areas where we have not gone so far.

Dr Huppert:I read out the quote from someone who has made the position clear when it comes to private companies carrying out public functions. Indeed, the code of practice has exactly the wording used in amendment 11, which the hon. Gentleman supported when we debated it on Tuesday. I do not want to take up too much of the Chairman’s kindness to discuss an issue that was rejected at that point, but it is happening as we wanted.

The matter was also touched upon a couple of days later in a Public Bill Committee on the Criminal Justice and Courts Bill (Official Report, Thursday 20 March 2014, 257-259) where accountability around public contracts delivered by private provides was being discussed:

Mr Slaughter: Absolutely not. I hope that the hon. Gentleman has read the article about Jago the rabbit that my hon. Friend the Member for Barnsley Central (Dan Jarvis) and I wrote for The Independent yesterday [It’s time we extended Freedom of Information to public services run by private companies – just ask Jago the Rabbit], which dealt with what should be done, which is to bring these companies within the ambit of FOI, and what the Minister of State did—with his usual skill, shall we say?—at Justice questions on Tuesday. He implied that that was what was going to happen, whereas in fact he was doing nothing more than putting round the line that the Cabinet Office has already indicated.

If I am wrong about that, I will give way in a moment and the hon. Gentleman can come back to me, but my understanding is that the Government—both parts of it, as long as they are just about coalescing—are of the view that the contracts that are drawn up should include this notional transparency. That is to say that they will encourage public authorities to encourage private companies to put clauses into contracts that will expose as much as possible, within the realms of commercial confidentiality. So the contracts will be open, with publication on websites and so forth of as much information about the contract as the two parties think fit. What we will not have is a duty on those private companies—in so far as they are carrying out public functions—to comply with the terms of the Freedom of Information Act, as would be the case in the public sector.

I accept that they are two sides of the same coin. On the one hand, of course it is a good idea that the information is made available voluntarily, but if it is not—either because the company does not choose to do so or because the contract is not drafted sufficiently well to ensure that it must—the citizen must have the right, through FOI, to require that information to be made available. As far as I am concerned, that is not what was said on Tuesday. I know that there is consultation going on, but if it is the intention of the Government—at least the Liberal Democrat part of the Government—to follow the line taken by my right hon. Friend the Member for Tooting (Sadiq Khan), the shadow Lord Chancellor, which he has repeated often in recent months, and require all those private companies performing public functions to come within the requirements of the Freedom of the Information Act, I would be pleased if the hon. Gentleman said so now.

Mr Slaughter:I take from that comment that even the hon. Gentleman does not understand what the Minister of State, Ministry of Justice, the right hon. Member for Bermondsey and Old Southwark, says, so opaque is it. If nobody, including the Minister, is going to answer my question, the answer will no doubt come out in the wash on a later occasion. However, it seems to me that that is not what is being promised. If it were, the Minister would be jumping up and claiming credit for it, but he is not. Therefore, I assume that that is not the case.

The significance of that is that those four companies about which I have just raised doubts—G4S, Serco, Capita, and we can safely add Atos—all told the Public Accounts Committee that they were prepared to accept the measures that the Committee proposed. It therefore appears that the main barrier to greater transparency lies within Government.

That is where we are. Even the companies that want to put themselves and the interests of their shareholders first are more keen on transparency and on answering the legitimate questions that are being asked by everyone— from ourselves to the chief inspector of prisons—than this Government are.

I say that because if we are to take this further leap down that path, it is only right that the Government do not just challenge, as the Minister has said, acknowledged frauds, but look at the entire performance behaviour, as well as the number of available companies that could step into the breach and deal with these matters.

What we must conclude from the conjunction of clauses 17 and 18 is that, first, the Government are prepared to take this leap in the dark, in terms of the reconfiguration of the youth estate and, secondly, that they are prepared to leave that entirely in the hands of the people who failed so many times in so many contracts, not least in running parts of the adult prison service.

For more on some of the specifics, see the House of Commons Public Accounts Committee report on “Contracting out public services to the private sector”, which for example recommended “that the Cabinet Office should explore how the FOI regime could be extended to cover contracts with private providers, including the scope for an FOI provision to be included in standard contract terms; that neither the Cabinet Office nor departments should routinely use commercial confidentiality as a reason for withholding information about contracts with private providers; [and that] The Cabinet Office should set out a plan for departments to publish routinely standard information on their contracts with private providers”.

There’s also a couple of related private members bills floating around at the moment – Grahame Morris’ empty Freedom of Information (Private Healthcare Companies) Bill 2013-14 “to amend the Freedom of Information Act 2000 to apply to private healthcare companies”, and Caroline Lucas’ Public Services (Ownership and User Involvement) Bill 2013-14 “to put in place mechanisms to increase the accountability, transparency and public control of public services, including those operated by private companies”. The latter >a href=”http://www.publications.parliament.uk/pa/bills/cbill/2013-2014/0160/cbill_2013-20140160_en_2.htm#l1g5″>proposes:

5 Transparency
(1) Where a relevant authority starts the process of procurement for a public services contract, it must make available to the public details of all bids received prior to the conclusion of the procurement exercise.
(2) Where a relevant authority enters into a public services contract, details of that contract shall be made available to the public within 28 days of the procurement decision.

6 Freedom of information
(1) The Secretary of State must designate as a public authority, pursuant to section 5(1)(b) of the Freedom of Information Act 2000, companies or other bodies which enter into a public services contract.
(2) “Public services contract” has the meaning contained within section 8 of this Act.
(3) The Secretary of State shall maintain a list of companies designated under section 6(1) of this Act.
(4) Requests under the Freedom of Information Act 2000 in respect of such companies or bodies can only be made in respect of information relevant to the provision of a public services contract.
(5) The Secretary of State must designate as a public authority, pursuant to section 5(1)(b) of the Freedom of Information Act 2000, any utility company subject to regulation by regulatory authorities as defined in section 8.

Finally, on the accountability and transparency thing, there’s a consultation on at the moment regrading “smaller authorities with an annual turnover not exceeding £25,000, including parish councils, [who] will be exempt from routine external audit” but instead will be subject to a transparency code (Draft transparency code for parish councils – consultation).

Related: Spending & Receipts Transparency as a Consequence of Accepting Public Money? If you accept public money for contracts that would otherwise be provided by a public service you should be subject to the same levels of FOI and transparency reporting. Why should public services have to factor this in to their bids for running a service when private companies don’t?

Other reading to catch up on: Commons Public Administration Select Committee [PASC] Report on Statistics and Open Data (evidence).

Written by Tony Hirst

April 6, 2014 at 10:25 am

Posted in opengov, Policy, Stirring

Tagged with , ,

More Open Data Frustrations – Unreadable Documentation from the DfE

One of the many things I’d like to spend my time doing is tinkering with data journalism doodles relating to local news stories. For example, via our local hyperlocal blog, I saw this post announced today: Isle of Wight has highest percentage of secondary school absentee rates in country. The post included a link to a Department for Education page (Pupil absence in schools in England, including pupil characteristics) containing links to the statistical release and the associated data sets:

dfes absentee data

Here’s what we get in the zipped datafile:

downloaded data

The school level dataset had the following column headings:

Year, country_code, country, GOR, GOR_code, LA, new_LA_code, LA_Name, URN, Estab, LAEstab, School_name, School_type, Academy_Flag, Academy_open_date, enrol_sum, SessionsPossible_sum, OverallAbsence_sum, AuthorisedAbsence_sum, UnauthorisedAbsence_sum, overall_absence_percent, auth_absence_percent, unauth_absence_percent, PA_15_sum, possible_sessions_pa_15_sum, overall_abs_pa_15_sum, auth_abs_pa_15_sum, unauth_abs_pa_15_sum, overall_absence_percent_PA_15, auth_absence_percent_PA_15, unauth_absence_percent_PA_15, sess_auth_illness, sess_auth_appointments, sess_auth_religious, sess_auth_study, sess_auth_traveller, sess_auth_holiday, sess_auth_ext_holiday, sess_auth_excluded, sess_auth_other, sess_auth_totalreasons, sess_auth_unclass, sess_unauth_holiday, sess_unauth_late, sess_unauth_other, sess_unauth_noyet, sess_unauth_totalreasons, sess_unauth_unclass, sess_overall_totalreasons

We can guess at what some of these refer to, but what, for example, do the “PA 15″ columns refer to? In this case, what we really should do is look up the actual definitions, which are described in the metadata description document; a document that just happens to be a Microsoft Word 2007 formatted document…

…a document that doesn’t play nicely either with the copy of Word I have on my Mac:

SOmetimesIPreferPDF

…or the converter that the Google docs uploader uses:

Google docs struggled too

In cases such as this, particularly where there are mathematical equations that often have very specific layout requirements, it can be “safer” to use a document format such as PDF that more reliably captures the appearance of the original page. (If we were really keen on reproducibility, we might also suggest that the equations were made available in an executable form, such as programme code or even as a spreadsheet (I’m not sure “Microsoft equations” are executable?).)

I gave myself a couple of hours to have a quick look through some of the data, but as it is I’ve spent an hour or so looking for ways of reading the metadata description document along with writing up my frustration around not being able to do so… Which is time spent not making sense of the data, or, indeed, its metadata…

PS in passing, I note the publication of the parliamentary Public Accounts Committee 37th report, Whole of Government Accounts 2010-11 again picks up on the way in which government data releases often fall short in terms of their usability (for example, this week MPs call for greater use of Whole of Government Accounts; see also last August Government must do better on transparency, say MPs).

PPS Here’s the solution I used in the end – Skydrive, Microsoft’s online storage/doc viewing play:

equations pt 1

equations pt 2

As it turns out, the equations could easily have been written using simple text strings…

PPPS as to the “15″ columns, the metadata files describes them along the following lines:

PA_15_sum Number of enrolments classed as persistent absentees (threshold of 15 per cent)

possible_session_pa_15_sum Sessions possible for persistent absentees (threshold of 15 per cent)

Which means what exactly?!

Written by Tony Hirst

April 11, 2013 at 5:48 pm

Posted in Anything you want, opengov

Tagged with

Public Sector Transparency – Do We Need Open Receipts Data as Well as Open Spending Data?

Some time ago, in the post Using Aggregated Local Council Spending Data for Reverse Spending (Payments to) Lookups, I described a way of looking at local council spending data based on how much different councils spent with each other.

This technique generalises within and across sectors, so for example we could look at how hospitals spend money with each other, or how police authorities spend money with each other. In this way, we can get a picture of how public bodies buy -and sell – services off each other. The mappings don’t have to relate to spend, either – we could equally well use this sort of model to see how hospitals transfer patients to one another, or how mental health or social care services offer out-of-area cover to each other, or how councils and housing trusts manage transfers between each other.

The insight that lets us produce this sort of view is that we have entities of a particular sort (hospitals, for example, or local councils), entering into transactions with other entities of the same sort. If these sorts of entity all operate under the same transparency rules, a requirement to publish outgoing (spend) transactions, for example, then we can recreate incoming (receipt) transactions from each entity of the same sort. For example, if local councils are required to publish details of spend over £x, then we can also learn how much councils received from other local councils by means of transactions over £x.

As the UK Government at least seems hell bent on getting markets established in the delivery of public services, markets that can include private companies, then we are faced with a possible asymmetry in transparency information.

UK Gov PolicyMaking local councils more transparent and accountable to local people

The public should be able to hold local councils to account about the services they provide. To do this, people need information about what decisions local councils are taking, and how local councils are spending public money.

And from the NHS:

NHS – Transparency of Spend

As part of the government’s commitment to greater transparency, there is a requirement to publish online each NHS organisation’s expenditure over £25,000. In accordance with the requirement NHS Direct publish this on the basis of payments made in each calendar month.

For example, if hospital A buys significant services off hospital B, and must report that spend under transparency legislation, we can build up a picture not only relating to A’s spend, but also B’s sale of services, because A’s data relating to spend with B is openly available; which means B’s receipts from A are also available. (In this example, if items can be itemised as less than £25k per item, then this form of reporting under transparency guidelines is not required.)

If hospital A now buys service of company C, then we can look up spend from hospital A to get a picture of how much public money is flowing out to the private sector and into company C. That is, we can get an idea of company C’s receipts from openly published hospital spending data. (Of course, games could be played with itemisation – 10 treatments at £3k a treatment would result in a ‘must declare’ spend of £30k on the course of treatment, but an undeclarable £3k per treatment if billing is organised that way.)

But what if company C buys services off hospital B (maybe even subcontracting services it was contracted to deliver by hospital A)? If the spend data of company C is not subject to transparency requirements, and the receipt data from the hospital is not publicly available, we lose sight of how money is being spent within and across the public service.

Whilst private companies may balk at being required to publish details of their own spending data, we might still be able to recreate a picture of their spend with public services by requiring public bodies to also publish receipts data, along with the current requirement to publish spend data?

Written by Tony Hirst

April 3, 2013 at 11:30 am

Posted in opengov, Policy

Tagged with

This Week in Open and Communications Data Land…

Following the official opening of the Open Data Institute (ODI) last week, a flurry of data related announcements this week:

Things have been moving on the Communications Data front too. Communications Data got a look in as part of the 2011/2012 Security and Intelligence Committee Annual Report with a review of what’s currently possible and “why change may be necessary”. Apparently:

118. The changes in the telecommunications industry, and the methods being used by people to communicate, have resulted in the erosion of the ability of the police and Agencies to access the information they require to conduct their investigations. Historically, prior to the introduction of mobile telephones, the police and Agencies could access (via CSPs, when appropriately authorised) the communications data they required, which was carried exclusively across the fixed-line telephone network. With the move to mobile and now internet-based telephony, this access has declined: the Home Office has estimated that, at present, the police and Agencies can access only 75% of the communications data that they would wish, and it is predicted that this will significantly decline over the next few years if no action is taken. Clearly, this is of concern to the police and intelligence and security Agencies as it could significantly impact their ability to investigate the most serious of criminal offences.

N. The transition to internet-based communication, and the emergence of social networking and instant messaging, have transformed the way people communicate. The current legislative framework – which already allows the police and intelligence and security Agencies to access this material under tightly defined circumstances – does not cover these new forms of communication. [original emphasis]

Elsewhere in Parliament, the Joint Select Committee Report on the Draft Communications Data Bill was published and took a critical tone (Home Secretary should not be given carte blanche to order retention of any type of data under draft communications data bill, says joint committee. “There needs to be some substantial re-writing of the Bill before it is brought before Parliament” adds Lord Blencathra, Chair of the Joint Committee.) Friend and colleague Ray Corrigan links to some of the press reviews of the report here: Joint Committee declare CDB unworkable.

In other news, Prime Minister David Cameron’s announcement of DNA tests to revolutionise fight against cancer and help 100,000 patients was reported via a technology angle – Everybody’s DNA could be on genetic map in ‘very near future’ [Daily Telegraph] – as well as by means of more reactionary headlines: Plans for NHS database of patients’ DNA angers privacy campaigners [Guardian], Privacy fears over DNA database for up to 100,000 patients [Daily Telegraph].

If DNA is your thing, don’t forget that the Home Office already operates a National DNA Database for law enforcement purposes.

And if national databases are your thing, there always the National Pupil Database which was in the news recently with the launch of a consultation on proposed amendments to individual pupil information prescribed persons regulations which seeks to “maximise the value of this rich dataset” by widening access to this data. (Again, Ray provides some context and commentary: Mr Gove touting access to National Pupil Database.)

PS A late inclusion: DECC announcement around smart meter rollout with some potential links to #midata strategy (eg “suppliers will not be able to use energy consumption data for marketing purposes unless they have explicit consent”). A whole raft of consultations were held around smart metering and Govenerment responses are also published today, including Government Response on Data Access and Privacy Framework, the Smart Metering Privacy Impact Assessment and a report on public attitudes research around smart metering. I also spotted an earlier consultation that had passed me by around the Data and Communications Company (DCC) License Conditions; here the response, which opens with: “The communications and data transfer and management required to support smart metering is to be organised by a new central communications body – the Data and Communications Company (“the DCC”). The DCC will be a new licensed entity regulated by the Gas and Electricity Markets Authority (otherwise referred to as “the Authority”, or “Ofgem”). A single organisation will be granted a licence under each of the Electricity and Gas Acts (there will be two licences in a single document, referred to as the “DCC Licence”) to provide these services within the domestic sector throughout Great Britain”. Another one to put on the reading pile…

Putting a big brother watch hat on, the notion of “meter surveillance” brings to mind BBC article about an upcoming (will hopefully thence be persistently available on iPlayer?) radio programme on “Electric Network Frequency (ENF) analysis”, The hum that helps to fight crime. According to Wikipedia, ENF is a forensic science technique for validating audio recordings by comparing frequency changes in background mains hum in the recording with long-term high-precision historical records of mains frequency changes from a database. In turn, this reminds me of appliance signature detection (identifying what appliance is switched on or off from its electrical load curve signature), for example Leveraging smart meter data to recognize home appliances. In context of audio surveillance, how about supplementing surveillance video cameras with microphones? Public Buses Across Country [US] Quietly Adding Microphones to Record Passenger Conversations.

Written by Tony Hirst

December 12, 2012 at 2:23 pm

Posted in Data, opengov, Policy

Tagged with , ,

The Opacity of Transparency

A letter from the Prime Minister to Cabinet Ministers on July 7th, 2011 stated that:

transparency boards will be established in each of the key delivery departments (health, education, justice, work and pensions, transport).

I’ve just done a quick trawl and found:

but not corresponding boards for DfE (Education) or MoJ (Justice)? If you know where to find any more info about these boards (or links to sources explaining why they don’t exist) please let me know via the comments…

It does, however, look as if there may be a Research Sector Transparency Board on the way…(?)

There’s also a smattering of other transparency boards/panels:

(Again, please let me know via the comments if I’m missing any…)

All departments are also required to publish open data strategies – you can find links to them here: Cabinet Office list of Departmental Open Data Strategies.

I do wonder what all this alleged transparency means or makes possible though…?

Written by Tony Hirst

August 29, 2012 at 3:35 pm

Posted in opengov, Policy

OpenData Reports Round Up (Links…)

It feels like there are just too many opendata reports being published at the moment to know which ones to read? They do potentially provide lots of possible content for structured reading exercises in an (open) data course though….?

Here’s a list of some the reports I’ve noticed recently, and that I haven’t really had time to read and digest properly:-(

Whilst not specifically about open data, these are also related to whole data and openness thang:

UK Gov Departments also published their open data strategies – they’re linked to from here: UK Gov Departmental Open Data Strategies.

PS I’m not sure if an English translation of this report (in Dutch) on Internal Business Models for Open Government Data is available anywhere?

Written by Tony Hirst

July 19, 2012 at 1:02 pm

Posted in opengov, Policy

Tagged with

The FOI Route to Real (Fake) Open Data via WhatDoTheyKnow

In FOI Signals on Useful Open Data?, I pondered whether we could make use of information about FOI to help identify what sorts of data folk might actually be interested in by virtue of making Freedom of Information (FOI) requests for that that data.

I couldn’t help but start to try working various elements of that idea through, so here’s a simple baby step to begin with – a scraper on Scraperwiki (Scaperwiki scraper: WhatDOTheyKnow requests) that searches for FOI requests made through WhatDoTheyKnow that got one or more Excel/xls spreasheets back as an attachment.

I’ve also popped up a Scraperwiki view that allows you to view data returning searches made to local councils or universities

Clicking through on an FOI request link takes you to the response that contains the data file, which can be downloaded directly or previewed on Zoho:

It strikes me that if I crawled the response pages, I could build my own index of data files, catalogued according to FOI request titles, in effect generating a “fake” data.gov.uk or data.ac.uk opendata catalogue as powered by FOI requests…? (What would be really handy in the local council requests would be if the responses were tagged with with appropriate LGSL code or IPSV terms (indexing on the way out) as a form of useful public metadata that can help put the FOI released data to work…?)

Insofar as the requests may or may not be useful as signaling particular topic areas as good candidates as “standard” open data releases, I still need to do some text analysis on the request titles. In the meantime, you can enter a keyword/key phrase in the Request text box in order to filter the table results to only show requests whose title contains the keyword/phrase. (The Council drop down list allows you to filter the table so that it only shows requests for a particular university/council.)

PS via a post on HelpMeInvestigate, I came across this list of FOI responses to requests made to the NHS Prescription Pricing Division. From a quick skim, some of the responses have “data” file attachments, though in the form of PDFs rather than spreadsheets/CSV. However, it would be possible to scrape the pages to at least identify ones that do have attachments (which is a clue they may contain data sets?)

So now I’m wondering – what other bodies produce full lists of FOI requests they have received, along with the responses to them?

PPS See also this gov.uk search query on FOI Release publications.

Written by Tony Hirst

April 28, 2012 at 10:42 pm

Posted in Data, opengov

Tagged with ,

Aggregated Local Government Verticals Based on LocalGov Service IDs

(Punchy title, eh?!) If you’re a researcher interested in local government initiatives or service provision across the UK on a particular theme, such as air quality, or you’re looking to start pulling together an aggregator of local council consultation exercises, where would you start?

Really – where would you start? (Please post a comment saying how you’d make a start on this before reading the rest of this post… then we can compare notes;-)

My first thought would be to use a web search engine and search for the topic term using a site:gov.uk search limit, maybe along with intitle:council, or at least council. This would generate a list of pages on (hopefully) local gov websites relating to the topic or service I was interested in. That approach is a bit hit or miss though, so next up I’d probably go to DirectGov, or the new gov.uk site, to see if they had a single page on the corresponding resource area that linked to appropriate pages on the various local council websites. (The gov.uk site takes a different approach to the old DirectGov site, I think, trying to find a single page for a particular council given your location rather than providing a link for each council to a corresponding service page?) If I was still stuck, OpenlyLocal, the site set up several years ago by Chris Taggart/@countculture to provide a single point of reference for looking up common adminsitrivia details relating to local councils, would be the next thing that came to mind. For a data related query, I would probably have a trawl around data.gov.uk, the centralised (but far form complete) UK index of open public datasets.

How much more convenient it would be if there was a “vertical” search or resource site relating to just the topic or service you were interested in, that aggregated relevant content from across the UK’s local council websites in a single place.

(Erm… or maybe it wouldn’t?!)

Anyway, here are a few notes for how we might go about constructing just such a thing out of two key ingredients. The first ingredient is the rather wonderful Local directgov services list:

This dataset is held on the Local Directgov platform which provides the deep links into Local council websites for a number of services in Directgov. The Local Authority Service details holds the local council URLS for over 240 services where the customer can directly transfer to the appropriate service page on any council in England.

The date on the dataset post is 16/09/2011, although I’m not sure if the data file itself is more current (which is one of the issues with data.gov.uk, you could argue…). Presumably, gov.uk runs off a current version of the index? (Share…. ;-) Each item in the local directgov services list carries with it a service identifier code that describes the local government service or provision associated with the corresponding web page. That it, each URL has associated with it a piece of metadata identifying a service or provision type.

Which leads to the second ingredient: the esd standards Local Government Service List. This list maps service codes onto a short key phrase description of the corresponding service. So for example, Council – consultation and community engagement is has service identifier 366, and Pollution control – air quality is 413. (See the standards page for the actual code/vocabulary list in a variety of formats…)

As a starter for ten, I’ve pulled the Directgov local gov URL listing and local gov service list into scraperwiki (Local Gov Web Pages). Using the corresponding scraper API, we can easily run a query looking up service codes relating to pollution, for example:

select * from `serviceDesc` where ToName like '%pollution%'

From this, we can pick up what service code we need to use to look up pages related to that service (413 in the case of air pollution):

select * from `localgovpages` where LGSL=413

We can also get a link to an HTML table (or JSON representation, etc) of the data via a hackable URI:

https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=htmltable&name=local_gov_web_pages&query=select%20*%20from%20%60localgovpages%60%20where%20LGSL%20%3D413

(Hackable in the sense we can easily change the service code to generate the table for the service with that code.)

So that’s the starter for 10. The next step that comes to my mind is to generate a dynamic Google custom search engine configuration file that defines a search engine that will search over just those URLs (or maybe those URLs plus the pages they link to). This would then provide the ability to generate custom search engines on the fly that searched over particular service pages from across localgov in a single, dynamically generated vertical.

A second thought is to grab those page, index them myself, crawl them/scrape them to find the pages they link to, and index those pages also (using something like tf-idf within each local council site to identify and remove common template elements from the index). (Hmmm… that could be an interesting complement to scraperwiki… SolrWiki, a site for compiling lists of links, indexing them, crawling them to depth N, and then configuring search ranking algorithms over the top of them… Hmmm… It’s a slightly different approach to generating custom search engines as a subset of a monolithic index, which is how the Google CSE and (previously) the Yahoo BOSS engines worked… Not scaleable, of course, but probably okay for small index engines and low thousands of search engines?)

Written by Tony Hirst

April 23, 2012 at 8:45 pm

Posted in onlinejournalismblog, opengov

Tagged with

First Sightings of the Data Strategy Board

Via a BIS press release earlier this week – Better access to public sector information moves a step closer – it seems that the Data Strategy Board is on its way, along with a Public Data Group and an Open Data User Group (these are separate from the yet to be constituted Open Standards Board (if you’re quick, the deadline for membership of the board is tomorrow: Open Standards Board – Volunteer Members and Board Advisers, – Ref:1238758) and its feeder Open Data Standards, and Open Technical Standards panels).

So what does the press release promise?

A new independently chaired Data Strategy Board (DSB) will advise Ministers on what data should be released [will this draw on data requests made to data.gov.uk, I wonder? - TH] and has the potential to unlock growth opportunities for businesses across the UK. At least one in three members of the DSB will be from outside government, including representatives of data re-users.

The DSB will work with the Public Data Group (PDG) – which consists of Trading Funds the Met Office, Ordnance Survey, Land Registry and Companies House – to provide a more consistent approach to improving access to public sector information. These organisations have already made some data available, which has provided opportunities for developers and entrepreneurs to create imaginative ways to develop or start up their own businesses based on high quality data.

Looking at the Terms of reference for the Data Strategy Board & the Public Data Group, we can broadly see how they’re organised:

Three departmental agendas then…?! A good sign, or, erm..?! (I haven’t read the Terms of reference properly yet – that’s maybe for another post…)

How these fit in with the Public Sector Transparency Board and the Local Public Data Panel, I’m not quite sure, though it might be quite interesting to try and map out the strong and weak ties between them once their memberships are announced? It’d also be interesting to know whether there’d be any mechanism for linking in with open data standards recommendations and development (via the Standards Hub process to ensure that as an when data gets released, there is at least an eye towards releasing it in a usable form!

The Government is making £7m available from April 2013 for the DSB to purchase additional data for free release from the Trading Funds and potentially other public sector organisations, funded by efficiency savings. An Open Data User Group, which will be made up of representatives from the Open Data community, will be directly involved in decisions on the release of Open Data, advising the DSB on what data to purchase from the Trading Funds and other public organisations and release free of charge.

So the DSB is a pseudo-cartel of sort-of government data providers (the Trading Funds) who are being given £7 million or so to open up data that the public purse (I think?) paid them to collect. The cash is there to offset the charges they would otherwise have made selling the data. (Erm… so, in order for those agencies to give their data away for free, we have to pay them to do it? Right… got it…) Presumably, the DSB members won’t be on the ODG who will be advising the DSB on what data to purchase from the Trading Funds and other public organisations and release free of charge (my emphasis). Note the explicit recognition here that free actually costs. In this case, public bodies are having data central gov paid them to collect bought off them by central gov so (central gov, or the bodies themselves) can then release it “for free”? Good. That’s clear then…

Francis Maude also clarifies this point: “The new structure for Open Data will ensure a more inclusive discussion, including private sector data users, on future data releases, how they should be paid for and which should be available free of charge.”

In addition: The DSB will provide evidence on how data from the Trading Funds – including what is released free of charge – will generate economic growth and social benefit. It will act as an intelligent customer advising Government on commissioning and purchasing key data and services from the PDG, and ensuring the best deal for the taxpayer. So maybe this means the Public Sector Transparency Board will now focus more on “public good” and transparency” arguments, leaving the DSB to demonstrate the financial returns of open data?

The Open Data User Group (ODUG) [will] support the work of the new Data Strategy Board (DSB). [The position of Chair of the group is currently being advertised, if you fancy it...: Chair of Open Data User Group, - Ref:1240914 -TH]. The ODUG will advise the DSB on public sector data that should be prioritised for release as open data, to the benefit of the UK.

As part of the process, an open suggestion site has been set up using the Delib Dialogue app to ask “the community” How should the Open Data User Group engage with users and re-users of Open Data?: [i]n advance of appointing a Chair and Members of the group, the Cabinet Office wants to bring together suggestions for how the ODUG should go about this engagement with wider users and re-users. We are looking for ideas about things like how the ODUG should gather evidence for the release of open data, how it should develop it’s advice to the DSB, how it should run its meetings and how it should keep the wider community up to date on developments (as well as other ideas you have).

A Twitter account has also been pre-emptively set up to manage some of the social media engagement activites of the group: @oduguk

The account currently has just over a couple of hundred followers, so I grabbed the list of all the folk they follow, then graphed folk followed by 30 or more current followers of @oduguk.

Here’s the graph, laid out in Gephi using a fore directed layout, with nodes colured according to modularity group and sized by eigenvector centrality:

Here’s the same graph with nodes size by betweenness centrality:

By the by, responses to the Data Policy for a Public Data Corporation consultation have also been published, including with the Government response, which I haven’t had chance to read yet… If I get a chance, I’ll try to post some thoughts/observations on that alongside a commentary on the terms of reference doc linked to above somewhere…

Written by Tony Hirst

March 21, 2012 at 1:54 pm

Posted in opengov

Tagged with , ,

Government Communications – Department Press Releases and Autodiscoverable Syndication Feeds

A flurry of articles earlier this week (mine will be along shortly) about the Data Strategy Board all broadly rehashed the original press release from BIS. Via the Cabinet Office Transparency minisite, I found a link to the press release via the COI News Distribution Service…

…whereupon I noticed that the COI – Central Office of Information – is to close at the end of this month (31 March 2012), taking with it the News Distribution Service for Government and the Public Sector (soon to be ex- of http://nds.coi.gov.uk/).

In its place is the following advice: “For government press releases please follow this link to find the department that you require http://www.direct.gov.uk/en/Dl1/Directories/A-ZOfCentralGovernment/index.htm This leads to a set of alphabetised pages with links to the various government departments… i.e. it points to a starting point for likely fruitless browsing and searching if you’re after aggregated press releases from gov departments.

(I’m not sure where News Sauce: UK Government Edition gets its data from, but if it’s by scrapes of departmental press releases rather than just scraping and syndicating the old COI content, then it’s probably the site I’ll be using to keep tabs on government press releases.)

FWIW, centralisation and aggregation are not the same in terms of architectures of control. Aggregation (then filter on the way out, if needs be) can be a really really useful way of keeping tabs on otherwise distributed systems… I had a quick look to see whether anyone was scraping and aggregating UKGov departmental press releases on Scraperwiki, but only came up with @pezholio’s LGA Press Releases scraper…

An easier way would be to hook up my feed reader to an OPML bundle that collected together RSS/Atom feeds of news releases from the various government websites. I’m not sure if such a bundle is available anywhere (if you know of one, please add a link in the comments below), but if: 1) gov departments do publish RSS/Atom feed containing their press releases; 2) they make these feeds autodiscoverable via their homepages, and: 3) ensure that said feeds are reliably identifiable as press release/media release feeds, it wouldn’t be too hard to build a simple OPML feed generator.

So for example, trawling through old posts, I note that the post 404 “Page Not Found” Error pages and Autodiscoverable Feeds for UK Government Departments used a Yahoo Pipes pipe to try to automatically audit feed autodiscovery on UK gov departmental homepages, though it may well have rotted by now. If I was to fix it, I’d probably reimplement it in Scraperwiki, as I did with my UK HEI feed autodiscovery thang (UK university autodiscoverable RSS Feeds (Scraperwiki scraper), and Scraperwiki View; about: Autodiscoverable Feeds and UK HEIs (Again…)). If you beat me to that, please post a link to your scraper below;-)

I have to admit I haven’t checked the state of feed autodiscovery on UK gov, local gov, or university websites recently. Sigh… another thing to add to the list of ‘maybe useful’ diversions…;-)

See also: Public Data Principles: RSS Autodiscovery on Government Department Websites?

PS This tool may or may not be handy if feed autodiscovery is new to you? Feed Autodiscovery in Javascript

PPS hmm, from Tracking Down Local Government Consultation Web Pages, I recall there are LGD service ID codes that lists identifiers for local government services that can be used to tag webpages/URLs on local government sites. Are there service identifiers for central government communication services (eg provision of press releases?) that could be used to find central gov department press releases (or local gov press releases for that matter?) Of course, if departments all had autodiscoverable press release feeds on their homepages, it’d be a more weblike way;-)

Written by Tony Hirst

March 21, 2012 at 12:14 pm

Posted in opengov

Tagged with , ,

Follow

Get every new post delivered to your Inbox.

Join 725 other followers