OUseful.Info, the blog…

Trying to find useful things to do with emerging technologies in open education

Visualising Traffic Count Data from transport.data.gov.uk

In a couple of posts last year (Hackable SPARQL Queries: Parameter Spotting Tutorial and First Dabblings With Pipelinked Linked Data) I started to explore how it might be possible to use Yahoo Pipes as an environment for sharing – and chaining together (albeit inefficiently) – queries to the data.gov.uk open transport data datastore.

Those posts concentrated on querying the datastore in order to find the location of traffic monitoring points according to various search criteria. In this post, I’ll show you one way of visualising traffic count data from a traffic count point using Many Eyes Wikified.

The first thing we need to do is come up with a query that will pull traffic monitoring data back from the transport datastore. My first point of call for finding a query to get me started is usually to search over the data.gov.uk Google group archive in my mailbox. As ever, @jenit had posted a ‘getting started’ solution:-)
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#&gt;
PREFIX traffic: <http://transport.data.gov.uk/0/ontology/traffic#&gt;
PREFIX geo: <http://www.w3.org/2003/01/geo/wgs84_pos#&gt;
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#&gt;

SELECT ?direction (SUM(?value) AS ?total)
traffic:count ?count .
?count a traffic:Count ;
traffic:category <http://transport.data.gov.uk/0/category/bus&gt; ;
traffic:direction ?direction ;
traffic:hour ?hour ;
rdf:value ?value .
REGEX(str(?hour), "^2008-")
GROUP BY ?direction

I tweaked this a little (using guesswork and pattern matching, rather than understanding, Chinese Room style;-) to come up with a tweaked query that appears to pull out traffic count data for different categories of vehicle on a particular day from a particular monitoring point:

SELECT ?vehicle ?direction ?hour (SUM(?value) AS ?total)
traffic:count ?count .
?count a traffic:Count ;
traffic:category ?vehicle ;
traffic:direction ?direction ;
traffic:hour ?hour ;
rdf:value ?value .
REGEX(str(?hour), "^2008-05-02T") )
GROUP BY ?vehicle ?direction ?hour

I then turned this into a Yahoo Pipes query block, using the date and traffic monitoring point as input parameters:

Here’s how to do that:

Having got the query into the pipes ecosystem, I knew I should be able to get the data out of the pipe as JSON data, or as a CSV feed, which could then be wired into other web pages or web applications. However, to get the CSV output working, it seemed like I needed to force some column headings by defining attributes within each feed item:

To tidy up the output a little bit more, we can sort it according to time and day and traffic count by vehicle type:

It’s then easy enough to grab the CSV output of the pipe (grab the RSS or JSON URI and just change the &_render=rss or &_render=json part of the URI to &_render=csv) and wire it into somewhere else – such as into Many Eyes WIkified:

Doing a couple of quick views over the data in Many Eyes wikified, it seemed as if there was some duplication of counting, in that the numbner of motor vehicles appeared to be the sum of a number of more specific vehicle types:

Data in Many Eyes WIkified

Looking at the car, van, bus, HGV and motorbike figures we get:

SUmmed totals

So I made the judgement call to remove the possibly duplicate motor-vehicle data from the data feed and reimported the data into Many Eyes WIkified (by adding some nonsense characters (e.g. &bleurghh to the feed URI so that Many Eyes thought it was a new feed.)

It was then easy enough to create some interactive visualisations around the traffic data point. SO for example, here we have a bubble chart:

Do you spot anything about traffic flow going North versus South at 8am compared to 5pm?

Lets explore that in a little more detail with a matrix chart:

Traffic data matrix chart http://wikified.researchlabs.ibm.com/ousefuldatagovuk/TrafficDataPoints:TrafficBubble

This shows us the relative counts for different vehicle types, again by time of day. Notice the different distribution of van traffic compared to car traffic over the course of the day.

A treemap gives a slightly different take on the same data – again, we can see how there is a difference between North and South flowing volumes at different times of day within each category:

One thing that jumps out at me from the treemap is how symmetrical everything seems to be at noon?!

All the above visualisations are interactive, so click through on any of the images to get to the interactibve version (Java required).

As to how to find traffic monitoring point IDs – try this.

PS a disadvantage of the above recipe is that to generate a visualisation for a different traffic point, I’d need to use the desired parameters when grabbing the CSV feed from the pipe, and then create new Many Eyes Wikified data pages and visualisation pages. However, using nothing more than a couple of web tools, I have managed to prototype a working mockup of a visualisation dashboard for traffic count data that could be given to a developer as a reference specification for a “proper” application. And in the meantime, it’s still useful as a recipe… maybe?

PPS While I was playing with this over the weekend, it struck me that if school geography projects ever do traffic monitoring surveys, it’s now possible for them to get hold of “real” data. If there are any school geography teachers out there who’d like to bounce around ways of making this data useful in a school context, please get in touch via a comment below :-)

Written by Tony Hirst

January 18, 2010 at 11:29 am

6 Responses

Subscribe to comments with RSS.

  1. Hi Tony

    Traffic data can be useful in both a practical sense (what will the impact be of a new development?) and in a more theoretical/academic sense.

    Quite how useful your above interpretations are I haven’t decided yet, but they’re certainly different from the ‘traditional’ (read unimaginative) ways of presenting traffic data. In particlar I don’t normally see ‘tidal flows’ displayed as clearly as in the TrafficBar.

    I’ve requested an account so I can take a look at the data and have a play.

    Richard Sweet

    January 20, 2010 at 8:53 pm

  2. Nice blog entry. I’m having trouble finding the data.gov.uk google group, do you have a URL for it please?


    Martin waller

    January 21, 2010 at 10:01 pm

  3. @martin no real need any more – just got to data.gov.uk and join the community there….

    @richard “Traffic data can be useful in both a practical sense (what will the impact be of a new development?)”

    so what sort of analysis do you do at the moment? where do you currently get traffic count data from and in what form?

    “Quite how useful your above interpretations are I haven’t decided yet”

    I just made them up – showing what a half hour hack and point and click could do…;-) Would love to be able to chat about what might actually be useful…

    “but they’re certainly different from the ‘traditional’ (read unimaginative) ways of presenting traffic data. In particlar I don’t normally see ‘tidal flows’ displayed as clearly as in the TrafficBar.”
    So how is the data currently presented; and what’s the workflow, eg ito acquiring the data, manipulating it (how?) and then: a)putting it into a reportable form (how? what sort of report/graphics/stats?), b) for whom, and c) so thy can them make what sort of decision.

    Would love to be able to hear tales of real world context/cases of how data like this is currently used, and what sort of value might be locked in it that isn’t currently exposed/extracted.

    Tony Hirst

    January 21, 2010 at 11:27 pm

  4. I think that’s my problem, I don’t see anywhere on the data.gov.uk website where I can register to join the forums etc, If I had data.gov.uk username it will allow me to change my password but getting a username in the first place is proving difficult?

    Many thanks for the last reply.


    Martin Waller

    January 23, 2010 at 1:03 pm

  5. Is there anywhere on the data.gov.uk website that actually has links to the various ontologies? The only one I’ve found that actually resolves is School, otherwise it seems to be a case of hoping someone’s posted a query involving what you want to do on the web and guessing. There seems to have been an era when this information was available, in late 2009, but now all the ontology URIs are 404s.


    July 7, 2010 at 10:18 am

  6. yorksranter, data.gov.uk are working on refreshing some of the datasets and the ontologies are being updated as well. Once the refresh has gone live, the ontologies should be published too.

    If not, it’s a bug, so please feel free to bug the data.gov.uk team :)

    As a workaround, you can always run queries like SELECT DISTINCT ?c WHERE {[] a ?c} to get a list of classes. Also, there’s general consensus that the ontologies should themselves be loaded into the triple-stores so you should be able to run (with the correct prefixes) queries like SELECT DISTINCT ?p WHERE {?p a rdf:Property}, or owl:ObjectProperty, owl:DatatypeProperty, etc.


    July 14, 2010 at 9:43 am

Comments are closed.


Get every new post delivered to your Inbox.

Join 1,347 other followers

%d bloggers like this: