What did you notice for the first time today?

A week late on posting this, catching up with Brian’s notes on the ILI 2013: Future Technologies and Their Applications Workshop workshop we ran last week, and his follow up – What Have You Noticed Recently? – inspired by not properly paying attention to what I had to say, here are few of my own reflections on what I heard myself saying at the event, along with additional (minor) comments around the set of ‘resource’ slides I’d prepped for the event, though I didn’t refer to many of them…

  • slides 2-6 – some thoughts on getting your eye into some tech trends: OU Innovating Pedagogy reports (2012, 2013), possible data-sources and reports;
  • slides 6-11 – what can we learn from Google Trends and related tools? A big thing: the importance of segmenting your stats; means are often meaningless. The Mothers’ Day example demonstrates two signal causes (in different territories – i.e. different segments) for the compound flowers trend. The Google Correlate example show how one signal may lead – or lag – another. So the question: do you segment your library data? Do you look for leading or lagging indicators?
  • slides 12-18 – what role should/does/could the library play in developing the reputation of the organisation’s knowledge producers/knowledge outputs, not least as a way of making them more discoverable; this builds on the question of whose role it is to facilitate access to knowledge (along with the question: facilitate access for whom?)? – my take is this fits in the role librarians often take of organising an institution’s knowledge.
  • slides 19-27 – what is a library for? Supporting discovery (of what, by whom)? (Helping others) organise knowledge, and gain access to information? Do research?
  • slides 28-30 – the main focus of my own presentation during the main ILI2013 conference (I’ll post the slides/brief commentary in another post): if the information we want to discover is buried in data, who’s there to help us extract or discover the information from within the data?
  • slides 31-32 – sometimes reframing your perception of an organisation’s offerings can help you rethink the proposition, and sometimes using an analogy helps you switch into that frame of mind. So if energy utilities provide “warm house” and “clean, dry clothes” service, rather than gas or electricity, what shift might libraries adopt?
  • slides 33-39 – a few idle idea prompts around the question of just what is it that libraries do, what services do they provide?
  • slide 40 – one of the items from this slide caused a nightmare tangent! The riff started with a trivial observation – a telling off I received for trying to use the phone on my camera to take a photo of a sign saying “no cameras in the library”, with a photocopier as a backdrop (original story). The purpose of this story was two-fold: 1) to get folk into the idea of spotting anachronisms or situations where one technology is acceptable where an equivalent or alternative is not (and then wonder why/what fun can be had around that thought;-); 2) to get folk into wondering how users might appropriate technology they have to hand to make their lives easier, even if it “goes against the rules”.
  • slide 41 – a thought experiment that I still have high hopes for in the right workshop setting…! if you overheard someone answer a question you didn’t hear with the phrase “did you try the library?”, what might the question be? You can then also pivot the question to identify possible competitors; for example, if a sensible answer to the same question is “did you try Amazon?”, Amazon might be a competitor for the delivery of that service.
  • slide 42 – this can lead on from the previous slide, either directly (replace “library” with “Amazon” or “Google”), or as way of generating ideas about how else a service might be delivered.

Slide not there – a riff on the question of: what did you notice for the first time today? This can be important for trend spotting – it may signify that something is becoming mainstream that you hadn’t appreciated before. To illustrate, I’ve started trying to capture the first time I spot tech in the wild with a photo, such as this one of an Amazon locker in a Co-Op in Cambridge, or a noticing from the first time I saw video screens on the Underground.

As with many idea generating techniques, things can be combined. For example, having introduced the notion of Amazon lockers, we might then ask: so what use might libraries make of such a system, or thing? Or if such things become commonplace, how might this affect or influence the expectations of our users??

Libraries Near Me Map (Courtesy of LibraryThing)

Given your location as postcode, where can you find a list of libraries near you? As with many of these public information style questions, it can be quite hard trying to find a single general answer. A crude way is to enter a search map along the lines of libraries near POSTCODE into Google Maps, but the results that are returned aren’t necessarily that good… Alternatively, you can go to your local council website, and then do a search there for libraries; but the format in which results are provided can vary; and if you’re a developer, there’s no immediately obvious way of creating (or consuming) a service that will allow you to create a small embeddable map showing the location of libraries in the vicinity of a particular postcode area.

Putting a tweet out last week to ask if anyone knew of a “library lookup by postcode”, several people suggested Worldcat, but @lynncorrigan suggested LibraryThing Local. Hmmm… here’s what it offers:

LibraryThing Local

That is, we can search by postcode, limit results to book related venues within a particular distance, and then filter down by library type, getting both a list of results back as well as map view. On running a search, we also get a URL that contains the search term as a parameter:

http://www.librarything.com/local/place/mk7%206aa%2C%20uk

This was a good start, so I had a look around the LibraryThing APIs to see if an API was available for the local service. I couldn’t spot anything, so I sent @librarythingtim a tweet to check, can got a confirmation back that there was no API for that service… Hmmm…

At times like this, it’s often worth trying different searches, and clicking different links (such as the “All”, “Bookstore” and “Library” fitlers, as well as the distance setting), to see what happens to the page – whether the URL changes for instance. It’s also worth doing a View Source on the page to see if there are any immediately obvious calls to AJAX web services that bring content into the page on those times when a setting change doesn’t cause the page to reload (with or without the same URL), but you suspect that there may have been a call to a webservice somewhere.

In the case of LibraryThing Local, clicking the links and changing the settings changed the displayed results without appearing to reload the page, but I wasn’t sure whether the change was just based on data stored within the web page, or whether the information was being pulled in from somewhere else.

Which is where developer tools can come in useful… One tool I use every now and gain is a Firefox extension called Firebug. This debugging environment operates within Firefox, and among other things can be used to track calls that a web page makes back to a server, along with any arguments passed to the server, and any results returned from it. Clicking between the “All” and “Libraries” links suggested that corresponding data was being pulled in to the page from a LibraryThing server:

LibraryTHing local - ajax call

The call was being made to http://www.librarything.com/ajax_venuesNearUser.php and returned HTML data that could be displayed within the results listing:

LibraryThing Local - data return

Hmmm…

The next step was to see if I could construct a URL without and session identifying variables to pull similar data back, such as:
http://www.librarything.com/ajax_venuesNearUser.php?&q=mk76aa,uk&showvenue=2

This did indeed work. The HTTP call also suggested there was a parameter identified as d. Could this be for distance? I tried it, setting it to 2, then 3. It seemed to work:-)

The next step was to try to pull the results into some sort of environment where I could play with it. The original motivation for getting this data was so that I could add libraries to a map of UK Online centres, based on my 5 Minute Hack – UK Centres Online Map, which used Yahoo pipes to geocode UK Online Centre addresses and generate a KML feed listing them in the vicinity of a particular postcode… So Yahoo pipes it was to be…

As well as RSS feeds, Yahoo Pipes can ingest XML and JSON data, as well as HTML pages. My first thought was to try to import the LibraryThing Local data as XML… but for some reason the Pipes Fetch Data block failed to parse it. Trying the Fetch Page block also failed to work… Hmmm…

At this point, I cast my mind around for other tools that might do the job. Google Spreadsheets? No; if the library data was in a table or a list, we might be able to use the importHTML formula, but is isn’t… Maybe I could try to import the HTML as XML…? Hmmm… XML… How about I try using YQL to parse the :LibraryThing Local data, and then pull the result into the pipe?

YQL parsing LibraryThing Local data

Success:-)

Here’s the query I used:
select * from html where
url=”http://www.librarything.com/ajax_venuesNearUser.php?showvenue=2&q=MK7+6AA&d=3″ and xpath=’//div[@class=”venueItem”]’

Rather conveniently, there is a YQL block available in Yahoo Pipes that can be provided with a YQL query and will pull the result back into the Pipes environment. Now the pipework can begin… but what do we want it to do?

The recipe will follow similar lines to the UK Online centres map pipe: grab a postcode from the user (and the search radius for good measure); construct the LibraryThing Local URL to fetch the raw data; use YQL to get the data into the pipes environment, then identify the address of each library, geocode it, create a title and piece of descriptive text (if required) then output the who lot. Simple:-)

To start then, let’s construct the LibraryThing Local URL:

LibraryThing Local map - create the URL

The reason we use a Create URL block is so that the postcode is automatically escaped (e.g. if the user places space characters in the postcode when they enter it, these need handling appropriately).

Having got the URL, we need to insert it into a YQL query – I use a bit of string replacing magic for that;-)

LibrryThing Local map - create the YQL query

Here’s the base part of the YQL query:
select * from html where
url=”MAGIC” and xpath=’//div[@class=”venueItem”]’

So, now we’re in a position to grab the local library data from LibraryThing Local via YQL:

Library Thing Local map - results from YQL

In the results, we see several separate items, one for each Library, but within each item, there are two “subresults”. Fortunately, these are “regular”, in that the result(s) for each Library is structured in the same way. Knowing how to handle the subresults required a little bit of Pipes trickery:

LibraryThing Local map - parsing results

It’s worth just looking at what’s going on in that block for a moment, and comparing it to the results that were coming from the YQL processor…

LibraryThing Local map - parsing reasults

All that remains to do now is geocode the address of each library, and output the result:

LibraryThing Local map - gecoding addresses

Here’s the result:

LibraryThing laocl map - pipe preview

You can see the pipe here: http://pipes.yahoo.com/ouseful/librarylocations

For details on how to use the output of this pipe to create your own embeddable maps, see the second half of 5 Minute Hack – UK Centres Online Map, in particular how to take the KML output of the pipe and use it to display the results in an embeddable Google map.

PS via @ostephens libraries lookup on People’s Network: “you can specify postcode and radius & get results as XML or route to xslt”