OUseful.Info, the blog…

Trying to find useful things to do with emerging technologies in open education

Libraries Near Me Map (Courtesy of LibraryThing)

Given your location as postcode, where can you find a list of libraries near you? As with many of these public information style questions, it can be quite hard trying to find a single general answer. A crude way is to enter a search map along the lines of libraries near POSTCODE into Google Maps, but the results that are returned aren’t necessarily that good… Alternatively, you can go to your local council website, and then do a search there for libraries; but the format in which results are provided can vary; and if you’re a developer, there’s no immediately obvious way of creating (or consuming) a service that will allow you to create a small embeddable map showing the location of libraries in the vicinity of a particular postcode area.

Putting a tweet out last week to ask if anyone knew of a “library lookup by postcode”, several people suggested Worldcat, but @lynncorrigan suggested LibraryThing Local. Hmmm… here’s what it offers:

LibraryThing Local

That is, we can search by postcode, limit results to book related venues within a particular distance, and then filter down by library type, getting both a list of results back as well as map view. On running a search, we also get a URL that contains the search term as a parameter:

http://www.librarything.com/local/place/mk7%206aa%2C%20uk

This was a good start, so I had a look around the LibraryThing APIs to see if an API was available for the local service. I couldn’t spot anything, so I sent @librarythingtim a tweet to check, can got a confirmation back that there was no API for that service… Hmmm…

At times like this, it’s often worth trying different searches, and clicking different links (such as the “All”, “Bookstore” and “Library” fitlers, as well as the distance setting), to see what happens to the page – whether the URL changes for instance. It’s also worth doing a View Source on the page to see if there are any immediately obvious calls to AJAX web services that bring content into the page on those times when a setting change doesn’t cause the page to reload (with or without the same URL), but you suspect that there may have been a call to a webservice somewhere.

In the case of LibraryThing Local, clicking the links and changing the settings changed the displayed results without appearing to reload the page, but I wasn’t sure whether the change was just based on data stored within the web page, or whether the information was being pulled in from somewhere else.

Which is where developer tools can come in useful… One tool I use every now and gain is a Firefox extension called Firebug. This debugging environment operates within Firefox, and among other things can be used to track calls that a web page makes back to a server, along with any arguments passed to the server, and any results returned from it. Clicking between the “All” and “Libraries” links suggested that corresponding data was being pulled in to the page from a LibraryThing server:

LibraryTHing local - ajax call

The call was being made to http://www.librarything.com/ajax_venuesNearUser.php and returned HTML data that could be displayed within the results listing:

LibraryThing Local - data return

Hmmm…

The next step was to see if I could construct a URL without and session identifying variables to pull similar data back, such as:
http://www.librarything.com/ajax_venuesNearUser.php?&q=mk76aa,uk&showvenue=2

This did indeed work. The HTTP call also suggested there was a parameter identified as d. Could this be for distance? I tried it, setting it to 2, then 3. It seemed to work:-)

The next step was to try to pull the results into some sort of environment where I could play with it. The original motivation for getting this data was so that I could add libraries to a map of UK Online centres, based on my 5 Minute Hack – UK Centres Online Map, which used Yahoo pipes to geocode UK Online Centre addresses and generate a KML feed listing them in the vicinity of a particular postcode… So Yahoo pipes it was to be…

As well as RSS feeds, Yahoo Pipes can ingest XML and JSON data, as well as HTML pages. My first thought was to try to import the LibraryThing Local data as XML… but for some reason the Pipes Fetch Data block failed to parse it. Trying the Fetch Page block also failed to work… Hmmm…

At this point, I cast my mind around for other tools that might do the job. Google Spreadsheets? No; if the library data was in a table or a list, we might be able to use the importHTML formula, but is isn’t… Maybe I could try to import the HTML as XML…? Hmmm… XML… How about I try using YQL to parse the :LibraryThing Local data, and then pull the result into the pipe?

YQL parsing LibraryThing Local data

Success:-)

Here’s the query I used:
select * from html where
url=”http://www.librarything.com/ajax_venuesNearUser.php?showvenue=2&q=MK7+6AA&d=3″ and xpath=’//div[@class="venueItem"]‘

Rather conveniently, there is a YQL block available in Yahoo Pipes that can be provided with a YQL query and will pull the result back into the Pipes environment. Now the pipework can begin… but what do we want it to do?

The recipe will follow similar lines to the UK Online centres map pipe: grab a postcode from the user (and the search radius for good measure); construct the LibraryThing Local URL to fetch the raw data; use YQL to get the data into the pipes environment, then identify the address of each library, geocode it, create a title and piece of descriptive text (if required) then output the who lot. Simple:-)

To start then, let’s construct the LibraryThing Local URL:

LibraryThing Local map - create the URL

The reason we use a Create URL block is so that the postcode is automatically escaped (e.g. if the user places space characters in the postcode when they enter it, these need handling appropriately).

Having got the URL, we need to insert it into a YQL query – I use a bit of string replacing magic for that;-)

LibrryThing Local map - create the YQL query

Here’s the base part of the YQL query:
select * from html where
url=”MAGIC” and xpath=’//div[@class="venueItem"]‘

So, now we’re in a position to grab the local library data from LibraryThing Local via YQL:

Library Thing Local map - results from YQL

In the results, we see several separate items, one for each Library, but within each item, there are two “subresults”. Fortunately, these are “regular”, in that the result(s) for each Library is structured in the same way. Knowing how to handle the subresults required a little bit of Pipes trickery:

LibraryThing Local map - parsing results

It’s worth just looking at what’s going on in that block for a moment, and comparing it to the results that were coming from the YQL processor…

LibraryThing Local map - parsing reasults

All that remains to do now is geocode the address of each library, and output the result:

LibraryThing Local map - gecoding addresses

Here’s the result:

LibraryThing laocl map - pipe preview

You can see the pipe here: http://pipes.yahoo.com/ouseful/librarylocations

For details on how to use the output of this pipe to create your own embeddable maps, see the second half of 5 Minute Hack – UK Centres Online Map, in particular how to take the KML output of the pipe and use it to display the results in an embeddable Google map.

PS via @ostephens libraries lookup on People’s Network: “you can specify postcode and radius & get results as XML or route to xslt”

Written by Tony Hirst

July 27, 2010 at 12:43 pm

2 Responses

Subscribe to comments with RSS.

  1. [...] For UK Onine Centres, it’s easy enough (5 Minute Hack – UK Centres Online Map); and there’s the start of a hack for plotting out the locations of all HEI libraries via SCONUL data (UK HE LIbraries map); but when I asked the lazyweb this question with respect to public libraries at the end of last week, the best response I got back was to use Librarything (Libraries Near Me Map (Courtesy of LibraryThing)). [...]

  2. I really do like your “showing the process” posts.

    hapdaniel

    August 10, 2010 at 7:46 pm


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 729 other followers

%d bloggers like this: