OUseful.Info, the blog…

This is something I’ve been meaning to do for ages, so spurred on by Martin Hawksey’s wonderful Google Gadgets port of my ad hoc Twitter network visulisation thing using Protovis (which Martin points out doesn’t work with IE9), I finally got round to it today: a wiring up of the OU modules Linked Data to the protovis app:

The data is pulled in from the OU Linked Data endpoint via Sparqlproxy (which provides a JSON output from the query that I can pull directly into the web page).

The query I’m using looks for courses related to the course of interest, and the courses related to those courses:

PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
select distinct ?name1 ?code2 ?name2 ?code3 ?name3 from <http://data.open.ac.uk/context/course> where {
?x a <http://purl.org/vocab/aiiso/schema#Module>.
?x <http://data.open.ac.uk/saou/ontology#courseLevel> <http://data.open.ac.uk/saou/ontology#undergraduate>.
?x <http://courseware.rkbexplorer.com/ontologies/courseware#has-title> ?name1.
?x <http://purl.org/goodrelations/v1#isSimilarTo> ?z.
?z <http://courseware.rkbexplorer.com/ontologies/courseware#has-title> ?name2.
?x <http://purl.org/vocab/aiiso/schema#code> 'T215'^^xsd:string.
?z <http://purl.org/vocab/aiiso/schema#code> ?code2.
?z <http://purl.org/goodrelations/v1#isSimilarTo> ?zz.
?zz <http://courseware.rkbexplorer.com/ontologies/courseware#has-title> ?name3.
?zz <http://purl.org/vocab/aiiso/schema#code> ?code3.
} LIMIT 100

(The endpoint is data.open.ac.uk/query; the explicit ‘T215′ course code identifier is paramterised in the URI that runs the query through Sparqlproxy.)

There’s all sorts of opportunities for coloring the nodes (eg to distinguish between the focal point course, it’s direct neighbours, and the neighbours of those neighbors) but that’s an exercise for another day. I should probably have a go at labeling them sensibly too…

(The ability to drag nodes around within the graph has also been added (back) – Martin noticed the order of a couple of the Protovis commands influenced whether this worked or not. Being able to relayout the chart reminds me how rubbish the force layout algorithm Protovis uses actually is!)

Drawing on Martin’s work (i.e. directly pinching his Google Gadget definition!) I also created a widget/gadget (XML) that lets you view the network of courses around a course in your own page…

Here’s the config page:

Of course, this being a WordPress.com hosted blog, I donlt think I can directly embed the gadget to prove that it works…

Related:
- data.open.ac.uk Linked Data Now Exposing Module Information
- Getting Started With data.open.ac.uk Course Linked Data
- Open University Undergraduate Module Map

PS to do – a reimagining of this, probably using arbor.js, where we just do the direct neigbours of a course code, but allow nodes to be clickable so that additional nodes and edges can be added to the graph dynamically… It might also be interesting to support search by keywords, and display courses that match keywords (in one colour) as well as related courses (in another), along with edges showing which courses are related…?

Picking up on data.open.ac.uk Linked Data Now Exposing Module Information, which describes how to query the Open University linked data store for course (that is, module) information, I’ve just posted an SVG map of how all the current* OU undergraduate course relate.

* if I understood @mdaquin correctly… specifically, a course is current if a description field is available for it.

You can see the graph here or by clicking through on the image below (if you’re using at least Firefox, Safari or Chrome, you should be able to click and drag the image to move it around, as well as zoom in and out.

The links between courses are the ‘related to’ links contained within the linked data. The nodes are sized according to degree and coloured according to modularity group, following application of the Gephi modularity statistic. The layout is an expanded form of a Yifan Hu layout.

The modularity statistic seems to identify clusters of courses reasonably well, allowing a student of potential student to get an overall view of the courses offered by the OU along with the courses that are naturally taken together. It may be interesting to explore the extent to which this sort of view may be used as a navigation surface, or made a little more interactive, for example by displaying information about a course (maybe including price and start date?!;-) when a user hovers over a course node.

Although not an OU/BBC co-pro, the “get some academics in to chat to Melvyn” format of BBC Radio 4′s In Our Time means that the OU has, over the years, had a handful of academics appearing on the programme. I’ve been mulling over opportunities for playing with the BBC programmes linked data (no RDF required) I wondered how easy it would be to grab the programmes that OU academics have appeared on. For example, it’s increasingly possible to see programmes associated with particular places (h/t to @gothwin for that; see his post on A Crude BBC Places Linked Data mashup for an application of that data) although the organisations listing is still a bit sparse.

Looking through the programme data, the participants in a programme are listed separately, but not their affiliations. However, in the free text that is used in the long synopsis of the programme, a convention exists to identify the guests, with affiliation or short bio, who appeared on that particular programme:

In the post Augmenting OU/BBC Co-Pro Programme Data With Semantic Tags, I described how the Thomson Reuters’ OpenCalais entity extraction/semantic tagging service could be used to augment the BBC programme data with additional data fields based on analysis of the supplied text. One of the extraction services identifies a set of related fields termed PersonCareer, which detail (where possible) the name of a person, their role, and the organisation they work for. The convention used to list the guests on each programme is appropriate for the extraction of PersonCareer data, at least in some cases.

Rather more reliable is the extraction of University names as Facility data types. What this means is that we can tag each programme with a list of Facilities relating to the universities represented by guests on the programme, and then – where a PersonCareer is extracted, attempt to text match the PersonCareer/Organization name with the extracted Facility name. (Sample code is available here. I had “issues” with character encodings, so there is an element if hackery involved:-( In order to aggregate data from across programmes in the series, I built up a network of programmes and participating institutions using a NetworkX representation, which then gets dumped to output files in a variety of graph formats.)

Here’s an example of the output, filtered to show programmes and programme tags (from the BBC data, rather than Calais extracted tags) that had some sort of association with the Open University:

The above diagram is actually a filtered view over the whole programme’n'university representation network using the Gephi ego network filter:

Node sizing is related to degree in this sub-network, and nodes are coloured according to node type (person, institution, tag, programme.) The graph shows programmes that an OU academic appeared on, and (where possible) which OU academic, by name. Programme tags from the BBC programme data are also shown, as are other institutions that appeared on the same programmes as the OU.

Here’s a snapshot of the full graph – you’ll notice there is some mismatch* in references between the universities mentioned that could possibly be reconciled using a string similarity technique or maybe running the data through Google Refine and using one or more of its string similarity/reconciliation tools.

* things are actually even more pathological: in some cases, I think that Oxbridge Colleges may be identified in PersonCareer metadata as the career organisation, rather than the university affiliation, which may well have been recognised as a Facility. If an organisation identified in a PersonCareer is not one of the Facilities added that has been identified and added to the graph, the organisation is also added. The question we’re left with is: do the errors such as they are make this graph, such as it is, completely use less, or is it better than nothing and something we can work with and improve incrementally as and how we can. [UPDATE: related maybe? Making Linked Data work isn’t the problem]

I’m not sure what the next step should be, but linking the OU ego-graph into the OU Linked Data would be one way forward. For example, displaying papers in ORO authored by appearing academics, or trying to relate programmes to related courses on the OU course catalogue (or even though not indexed in the OU Linked Data store, courses on OpenLearn). A big problem with brokering the Linked Data connections is that I’d have to do free text/regular expression searches on the OU Linked Data store using terms from the BBC/OpenCalais data. THat is, there are no common unique identifier/URIs that can be used as “proper” linking terms:-(