Tag: sparqlproxy

Programming Pipes With Delicious and Sharing data.gov.uk SPARQL Queries As A Result

In the post Sharing Linked Data Queries With Mortals… I described a rather clunky pattern for sharing SPARQL queries onto a data.gov.uk sparql endpoint using delicious, along with a Yahoo pipe to generate a SPARQLProxy URI that identified a CSV formatted output from the query that could be consumed in something like Google spreadsheets (e.g. Viewing SPARQLed data.gov.uk Data in a Google Spreadsheet).

In this post, I’ll refine that pattern a little more and show how to use delicious to bookmark a “processed” form of the output of the query, along with all the ingredients needed to generate that output. In a later post (hopefully before Christmas) I’ll try to show how the pattern can be used to share queries into other datastores, such as Google visualization API queries into a Google spreadsheet.

[The post describes two independent operations – firstly, how to generate a sparqlproxy uri from a set of delicious tags; secondly, how to generate a map from sparqlproxy output.]

In what follows, I’m using delicious as a database, and delicious tags as machine tags that can be used as arguments within a Yahoo pipe. The intention is not to suggest that this is even a good way of sharing SPARQL queries and demo outputs, but it does show an improvised way of how to share them,. It also provides just enough raw material to allow UI designers to think how we might capture, share and display sample use cases that go from SPARQL query to human meaningful output.

So here’s what a “finished” delicious bookmark looks like:

The three “machine tags”:
– endpoint:http://services.data.gov.uk/transport/sparql
– output:csv
– query:http://codepad.org/hPo2XIzx/raw.txt
are used as instructions in a Yahoo pipe that generates a query using SPARQLProxy:

A feed of bookmarked queries is pulled in from delicious, and checked to see that all the programming arguments that are required are there. (The sparlproxy_demo tag requirement is just a convenience.)

If all the arguments are available, their values are captured and passed to a routine to construct the SPARQLProxy URIs:

The query bookmarked by the tags is a query onto the transport database that pulls out the location traffic monitoring points on thee M5. The bookmarked URI is a demonstration of how to use the output of that query. In the current example, the bookmarked demo URI looks like this:

http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=
&q=http:%2F%2Fpipes.yahoo.com%2Fpipes%2Fpipe.run%3F_id%3D1c77faa95919df9dbea678a6bf4881c6%26_render%3Dkml
&sll=37.0625,-95.677068&sspn=44.25371,70.751953&ie=UTF8&z=8

That is, it is a bookmark to a Google map that is displaying a KML feed pulled in from a Yahoo pipe.

This is the Google Map:

This is the Yahoo pipe the KML is pulled from:

And this is a peek inside that pipe:

The URI for the fetched CSV file is on that was generated from the bookmarked query tags by the first pipe shown above.

So to recap – this bookmark:

links through to a Google map showing traffic monitoring locations on the M5. The map is an ‘end-user’ demo that shows how Government data may be displayed in a meaningful way. In addition, th tags in the bookmark carry enough information for the user to construct a SPARQL qury that will generate the data that is displayed by the map. A utility Yahoo pipe (ID 22059ac6f967d7117b1262586c92b371) can take a delicious RSS feed containing the bookmark and generate a SPARQLProxy URI that calls the tagged endpoint with the tagged query and generates the tag specified output. There is then a break in the chain. A utility pipe capable of handling SPPARQLProxy generated CSV from data.gov.uk transport data generates a KML version of the data, which is then passed to a Google map.

So what? So I don’t know what… but it has got me thinking that what might be useful is a quick way of sharing:
– info that is sufficient to generate and run a particular SPARQL query.
– along with a link to an ‘end user’ demo showing how that data might be used or displayed
– all in one handy package…

Which is what the above does… sort of.. though it’s still way too complicated for mortals…

PS It occurs to me that it might be possible to define a pipeline using delicious tags too? Maybe something like:
pipeline:22059ac6f967d7117b1262586c92b371?name=psychemedia&tag=transportDemo//
1c77faa95919df9dbea678a6bf4881c6?_render=kml//
gmap

where the first step in the pipeline (2059ac6f967d7117b1262586c92b371?name=psychemedia&tag=transportDemo) says “run the delicious RSS feed from psychemedia/transportDemo (assuming I had recursively bookmarked that record with transportDemo) through the pipe with ID 2059ac6f967d7117b1262586c92b371 to generate the SPARQLProxy URI, then pass that to the pipe with ID 1c77faa95919df9dbea678a6bf4881c6 which should generate a KML output, which in turn should be sent to a Google map?!

Sharing Linked Data Queries With Mortals…

So I know, I know, I b****y well know how important the whole RDF thing is for Linked Data, and I know we’re not there yet with respect to actual people using data pulled from data.gov.* sources, and I’m starting to be persuaded that maybe data.gov.uk is only there to feed the growth of semantic web developer capacity but ultimately, ultimately, it will probably be folk who can’t cope with anything other than a spreadsheet who are going to have to use this data…

…so the spirit in which this is offered is one of just trying to protect the interests of potential future end users while the geeky tech developer astronauts (astronauts being @iand’s term;-) do the groundwork’n’spadework and make design decisions whose full impact may not otherwise be realised for a little while yet…

So what am I offering…? A quick’n’dirty way of sharing bookmarks into the sparqlproxy Web Service that I posted about in Viewing SPARQLed data.gov.uk Data in a Google Spreadsheet.

So how does it work? Like this…

The geeky tech SPARQL speaking astronaut writes their SPAQRL query and posts it into codepad:

They grab the link to the raw text and bookmark it in delicious; the SPARQL endpoint for the query is pasted into the description, and a brief description of the query into the title; the required output is identified using an output: machine tag (e.g. output:, output:sparql, output:html, output:csv, output:xml, output:exhibit or output:gvds):

(An alternative might be to have the endpoint as the title, and the description as the description, or a brief description as a title, a full description asa description, and a endpoint: “machine tag” for the endpoint, but this was just a proof of concept, right? ;-)

The following pipe constructs the SPARQLProxy query for each bookmark using the specified query, endpoint and output type (at the moment, the pipe also requires a sparqlproxy_demo tag to be present):

A link to the result of the query, suitably transformed, is then rewritten as the link in the output feed.

A bit of tidying up on the pipe lets you specify a delicious user and/or tag as the origin of the bookmarked links…

So there we have it, an easy way to share SPARQLed queries and get access to “human usable” outputs…

PS there’s no reason, in the recipe above, not to also use the sparql endpoint URI in a tag or machine tag too, to allow for queries run over the same bookmark to be collected together and pulled out of delicious by tag/endpoint…