Trying to clear my head of code on a dog walk after a couple of days tinkering with the nomis API and I started to ponder what an API is good for.
Chris Gutteridge and Alex Dutton’s open data excuses bingo card and Owen Boswarva’s Open Data Publishing Decision Tree both suggest that not having an API can be used as an excuse for not publishing a dataset as open data.
So what is an API good for?
I think one naive view is that this is what an API gets you…
It doesn’t of course, because folk actually want this…
Which is not necessarily that easy even with an API:
For a variety of reasons…
Even when the discovery part is done and you think you have a reasonable idea of how to call the API to get the data you want out of it, you’re still faced with the very real practical problem of how to actually get the data in to the analysis environment in a form that is workable on in that environment. Just because you publish standards based SDMX flavoured XML doesn’t mean anything to anybody if they haven’t got an “import from SDMX flavoured XML directly into some format I know how to work with” option.
And even then, once the data is in, the problems aren’t over…
(I’m assuming the data is relatively clean and doesn’t need any significant amount of cleaning, normalising, standardising, type-casting, date par;-sing etc etc. Which is of course likely to be a nonsense assumption;-)
So what is an API good for, and where does it actually exist?
I’m starting to think that for many users, if there isn’t some equivalent of an “import from X” option in the tool they are using or environment they’re working in, then the API-for-X is not really doing much of a useful job for them.
Also, if there isn’t a discovery tool they can use from the tool they’re using or environment they’re working in, then finding data from service X turns into another chore that takes them out of their analysis context and essentially means that the API-for-X is not really doing much of a useful job for them.
What I tried to do in doodling the Python / pandas Remote Data Access Wrapper for the Nomis API for myself create some tools that would help me discover various datasets on the nomis platform from my working environment – an IPython notebook – and then fetch any data I wanted from the platform into that environment in a form in which I could immediately start working with it – which is to say, typically as a pandas dataframe.
I haven’t started trying to use it properly yet – and won’t get a chance to for a week or so at least now – but that was the idea. That is, the wrapper should support a conversation with the discovery and access parts of the conversation I want to have with the nomis data from within my chosen environment. That’s what I want from an API. Maybe?!;-)
And note further – this does not mean things like a pandas Remote Data Access plugin or a CRAN package for R (such as the World Bank Development Indicators package or any of the other data/API packages referenced from the rOpenSci packages list should be seen as extensions of the API. At worst, they should be seen as projections of the API into user environments. At best, it is those packages that should be seen as the actual API.
APIs for users – not programmers. That’s what I want from an API.
PS See also this response from @apievangelist: The API Journey.
Just a quick observation about how to grab the lists of circles an individual is in on Google+, and who’s in their circles… From looking at the HTTP calls that are made when you click on the ‘View All’ links for who’s in a person’s circles, and whose circles they are in, on their Google Profile page, we see URLs of the form:
– in X’s circles:
– in whose circles?
You can find the GOOGLEPLUSUSERID by looking in the URL of a person’s Google+ profile page. I’m not sure if the &rt=j is required/what it does exactly?
Results are provided via
JSON some crappy hacky weird format, with individual records of the form:
,["Liam Green-Hughes",,,,"4af0a6e759a1b","EIRbEkFnHHwjFFNFCJwnCHxA", "BoZjAo3532KEBnJjHkFxCmRz",,"//lh5.googleusercontent.com/-z4ZRcOrNx7I/AAAAAAAAAAI/AAAAAAAAAAA/cQBO1TSuucI/photo.jpg",,1,
"Milton Keynes",,,"Developer and blogger",0,,
A scratch API, of a sort, is available form http://html5example.net/entry/tutorial/simple-python-google-plus-api
Note that these connections don’t seem to be available via the Google Social Graph API? (Try popping in the URL to your Google+ profile page and see who it thinks you’re linked to…)
One of the sessions I attended at last year’s CETIS get together was the UKOLN organised Technological Innovation in a World of Web APIs session (see also My CETIS 2008 Presentations and What Makes A Good API? Doing The Research Using Twitter).
This session formed part of a project being co-ordinated by UKOLN’s homeworking Marieke Guy – JISC “Good APIs” project (project blog) – which is well worth getting involved with because it might just help shape the future of JISC’s requirements when they go about funding projects…
(So if you like SOAP and think REST is for wimps, keep quiet and let projects that do go for APIs continue to get away with proposing overblown, unfriendly, overengineered ones…;-)
So how can you get involved? By taking this survey, for one thing:
The ‘Good APIs’ project aims to provide JISC and the sector with information and advice on best practice which should be adopted when developing and consuming APIs.
In order to collate information the project team have written a very brief research survey asking you about your use of APIs (both providing and consuming).
I don’t know if the project will have a presence at the JISC “Developer Happiness” Days (the schedule is still being put together) but it’d be good if Marieke or Brian were there on one of the days (at least) to pitch in some of the requirements of a good API that they’ve identified to date;-)
PS here’s another fun looking event – Newcastle Maker Faire.