Searching for Curriculum Development Course Insights

For almost as long as I can remember (?! e.g. Search Powered Predictions), I’ve had the gut feeling that one of the most useful indicators about the courses our students want to study is their search behaviour, both in terms of searches that drive (potential) students to the OU courses and qualifications website from organic search listings, as well as their search behaviour whilst on the OU site, and whilst floundering around within the courses and quals minisite.

A quick skim through our current strategic priorities doc (OU Futures 2008 (internal only), though you can get a flavour from the public site: Open University Strategic Priorities 2007) suggests that there is increased interest in making use of data, for example as demonstrated by the intention to develop a more systematic approach for new curriculum developments, such that the student market, demography and employment sectors are the primary considerations.

So, to give myself something to think about over the next few days/weeks, here’s a marker post about what a “course search insights” tool might offer, inspired in part by the Google Youtube Insights interface.

So, using Youtube Insight as a starting point, let’s see how far we can get…

First off, the atom is not a Youtube video, it’s a course, or to be more exact, a course page on the courses and quals website… Like this page for T320 Ebusiness technologies: foundations and practice for example. The ideas are these: what might an “Insight” report look like for a course page such as this, how might it be used to improve the discoverability of the page (and improve appropriate registration conversion rates), and how might search behaviour inform curriculum development?

Firstly, it might be handy to segment the audience reports into four:

  • people hitting the page from an organic search listing;
  • people hitting the page from an internal (OU search engine) search listing;
  • people hitting the page from an ‘organic’ link on a third party site (e.g. a link to the course page from someone’s blog);
  • people hitting the page from an external campaign/adword etc on a search engine;
  • people hitting the page from any other campaign (banner ads etc);
  • the rest…

For the purposes of this post, I’ll just focus on the first two, search related, referrers… (and maybe the third – ‘organic’ external links). What would be good to know, and how might it be useful?

First off, a summary report of the most popular search terms would be handy:

– The terms used in referrers coming from external organic search results give us some insight into the way that the search engines see the page – and may provide clues relating to how to optimise the page so as to ensure we’re getting the traffic we expect from the search engines.

– The terms used within the open.ac.uk search domain presumably come from (potential) students who have gone through at least one micro-conversion, in that they have reached, and stayed in, the OU domain. Given that we can (sometimes) identify whether users are current students (e.g. they may be logged in to the OU domain as a student) or new to the OU, there’s a possibility of segmenting here between the search terms used to find a page by current students, and new prospects.

(Just by the by, I emailed a load of OU course team chairs a month or two ago about what search terms they would expect potential students to use on Google (or on the OU search engine) to find their course page on the courses and quals site. I received exactly zero responses…)

The organic/third party incoming link traffic can also provide useful insight as to how courses are regarded from the insight – an analysis of link text, and maybe keyword analysis of the page containing the link – can provide us with clues about how other people are describing our courses (something which also feeds into the way that the search engines will rank our course pages; inlink/backlink analysis can further extend this approach.). I’m guessing there’s not a lot of backlinking out there yet (except maybe from professional societies?), but if and when we get an affiliate scheme going, this may be one to watch…?

So that’s one batch of stuff we can look at – search terms. What else?

As a distance learning organisation, the OU has a national reach (and strategically, international aspirations), so a course insight tool might also provide useful intelligence about the geographical location of users looking at a particular course. Above average numbers of people reading about a course from a particular geo-locale might provide evidence about the effectiveness of a local campaign, or even identify a local need for a particular course (such as the opening or closure of large employer).

The Youtube Insight reports shows how as the Google monster gets bigger, it knows more and more about us (I’m thinking of the Youtube Insight age demographic/gender report here). So providing insight about the gender split and age range of people viewing a course may be useful (we can find this information out for registered users – incoming users are rather harder to pin down…), and may provide further insight when these figures are compared to the demographics of people actually taking the course, particularly if the demographic of people who view a course on the course catalogue page differs markedly from the demographics of people who take the course…

(Notwithstanding the desire to be an “open” institution, I do sometimes wonder whether we should actually try to pitch different courses at particular demographics, but I’m probably not allowed to say things like that…;-)

As well as looking at search results that (appear) to provide satisfactory hits, it’s also worth looking at the internal searches that don’t get highly relevant results. These searches might indicate weak optimisation of pages – appropriate search terms donlt find appropriate course pages – or they might identify topics or courses that users are looking for that don’t exist in the current OU offerings. Once again, it’s probably worth segmenting these unfulfilled/unsatisfactory courses according to new prospects and current students (and maybe even going further, e.g. by trying to identify the intentions of current students by correlating their course history with their search behaviour, we may gain insight into emerging preferences relating to free choice courses within particular degree programmes).

To sum up… Search data is free, and may provide a degree of ‘at arms length’ insight about potential students before we know anything about them ‘officially’ by virtue of them registering with us, as well as insight relating to emerging interests that might help drive curriculum innovation. By looking at data analysis and insight tools that are already out there, we can start to dream about what course insight tools might look like, that can be used to mine the wealth of free search data that we can collect on a daily basis, and turn it into useful information that can help improve course discovery and conversion, and feed into curriculum development.

Special Interest Custom Search Engines

A recent post by Downes (PubMed Now Indexes Videos of Experiments and Protocols in Life Sciences) reminded me of a Google custom search engine I started to put together almost a year or ago to provide a meta-search over science experiment protocols.

At the time, I managed to track three likely sites down, although despite my best intentions when I created the initial CSE, I haven’t managed even cursory maintenance of the site.

Anyway, for what it’s worth, here’s a link to my Science Experimental Protocols Video Search (a search for DNA will show you what sorts of results are typical). If you know of any other sites that publish scientific experimental protocols, please fee free to post a link in the comments to the post.

Another custom search engine I started looking at at the start of this year, inspired by a conversation with a solicitor friend over New Year, was a search of UK (English and Scottish) legislation. The intention here was to come up with a CSE that could provide a value adding vertical search site to a legal website. If i remember correctly (?!;-) the CSE only took an hour or so pull together, so even though we never pursued embedding it on live website, it wasn’t really that much time to take out…

If you want to check it out, you can find it here: LegalDemo.

One CSE I do maintain is “How Do I?”, a metasearch engine over instructional video websites. There are almost as many aggregating websites of this ilk as there are sites publishing original instructional content, but again, it didn’t take long to pull together, and it’s easy enough to maintain. You can find the search engine here: “How Do I?” instructional video metasearch engine, and a brief description of its origins here: “How Do I…” – Instructional Video Search.

Another 10 minute CSE I created, this time following a comment over a pint about the “official” OpenLearn search engine, was an OpenLearn Demo CSE (as described here: OpenLearn Custom Search).

And finally (and ignoring other the other half-baked CSEs I occasionally dabble with), there’s the CSE I’ve been doodling with most recently: the OUseful search engine (I need to get that sorted on a better URL..). This CSE searches over the various blogs I’ve written in the past, and write on at the moment. If you want to search over posts from the original incarnation of OUseful.info, this is one place to do it…

Just looking back over the above CSEs, I wonder again about who’s job it is (if anyone’s), to pull together and maintain vertical search engines in an academic environment, or show students how they can crate their own custom search engines? (And one level down from that, who’s role is it to lead the teaching of the “search query formulation” information skill?)

In the OU at least, the Library info skills unit have been instrumental in engaging with course teams to develop information literacy skills, as well as leading the roll out of Beyond Google… but I have to admit, I do wonder just how well equipped they are to helping users create linked data queries, SPARQL queries, or SQL database queries containing a handful of joins? (I also wonder where we’re teaching people how to create pivot tables, and the benefits of them…?!)

Thinking about advanced queries, and the sighs that go up when we talk about how difficult it is to persuade searchers to use more than two or three keyword search terms, I’ve also been wondering what the next step in query complexity is likely to be after the advanced search query. And it strikes me that the linked data query is possibly that next step?

Having introduced the Parallax Freebase interface to several people over the last week, it struck me that actually getting the most out of that sort of interface (even were Freebase populated enough for more than a tiny minority of linked queries to actually work together) is not likely to be the easiest of jobs, particularly when you bear in mind that it’s only a minority of people who know how to even conceptualise advanced search queries, let alone know how to construct them at a syntactic level, or even via a web form.

The flip side to helping users create queries is of course helping make information amenable to discovery by search, as Lorcan Dempsey picks up on in SEO is part of our business. Here again we have maybe another emerging role for …. I don’t know…? The library? And if not the library, then whom?

(See also: The Library Flip, where I idly wondered whether the academic library of the future-now should act “so as to raise the profile of information it would traditionally have served, within the search engine listings and at locations where the users actually are. In an academic setting, this might even take the form of helping to enhance the reputation of the IP produced by the institution and make it discoverable by third parties using public web search engines, which in turn would make it easy for our students to discover OU Library sponsored resources using those very same tools.”)

PS Just a quick reminder that there are several OU Library job vacancies open at the moment. You can check them out here: OU Library Jobs Round-Up (August 2008).

ORO Results in Yahoo SearchMonkey

It’s been a long – and enjoyable – day today (err, yesterday, I forgot to post this last night!), so just a quick placeholder post, that I’ll maybe elaborate on with techie details at a later date, to show one way of making some use of the metadata that appears in the ORO/eprints resource splash pages (as described in ORO Goes Naked With New ePrints Server): a Yahoo SearchMonkey ORO augmented search result – ORO Reference Details (OUseful).

The SearchMonkey extension – which when “installed” in your Yahoo profile, will augment ORO results in organic Yahoo search listings with details about the publication the reference appears in, the full title (or at least, the first few characters of the title!), the keyowrds used to describe the reference and the first author, along with links to a BibTeX reference and the document download (I guess I could also add a link in there to a full HTML reference?)

The SearchMonkey script comes in two parts – a “service” that scrapes the page linked to from the results listing:

And a “presentation” part, that draws on the service to augment the results:

It’s late – I’m tired – so no more for now; if you interested, check out the Yahoo SearchMonkey documentation, or Build your own SearchMonkey app.

What Google Thinks of the OU…

More and more search boxes now try to help the user out by making search completion recommendations if you pause awhile when typing query terms into a search box.

So here’s how you get helped out on Youtube:

And here’s what Google suggest is offering on a default (not signed in) Google personal page:

Here’s Yahoo:

Google Insights for Search also provides some food for thought from a free tool you can run against any search terms that get searched on enough. So here for example is the worldwide report for searches on open university over the last 90 days:

Tunneling down to look at searches for open university from the UK, I notice quite a lot were actually looking for information about university open days… Hmmm… do we have a permanent “open day” like web page up onsite anywhere, I wonder?

Let’s see – after all, the OU search engine never fails…

… to provide amusement…

Google comes up with:

Would it make sense, I wonder, to try to capitalise on the name of the university and pull traffic in to a landing page specifically designed to siphon off Google search traffic from students looking for open days at other universities? ;-)

“The Open University: where every day is a university open day. From Newcastle to Bristol, London to Leeds, Oxford to Cambridge, Birmingham to Edinburgh, Cardiff to Nottingham, why not pop in to your local regional Open University center to see what Open University courses might be right for you?”, or somesuch?! Heh heh… :-)

Google Personal Custom Search Engines?

A couple of days ago, I gave a talk about possible future library services, and in passing mentioned the way that my Google search results are increasingly personalised. Martin picked up on this in a conversation over coffee, and then in a blog post (“Your search is valuable to us“):

This made me think that your search history is actually valuable, because the results you get back are a product of the hours you have invested in previous searches and the subject expertise in utilising search terms. So, if you are an expert in let’s say, Alaskan oil fields, and have been researching this area for years, then the Google results you get back for a search on possible new oil fields will be much more valuable than the results anyone else would get.

[I]f you can assemble and utilise the expert search of a network of people, then you can create a socially powered search which is very relevant for learners. Want to know about really niche debates in evolution? We’ve utilised Richard Dawkins, Steve Jones and Matt Ridley’s search history to give you the best results. Or if you prefer, the search is performed as the aggregation of a specialist community.

There are more than a few patents in this area of course (you can get a feel for what the search engines are (thinking about) doing in this area by having a read through these SEO by the SEA posts on “search+history+personal”), but I was wondering:

how easy would it be to expose my personal search history reranking filter (or whatever it is that Google uses) as a custom search engine (under my privacy controls, of course)?

As Martin says (and as we discussed over coffee), you’d want to disable further personalisation of your CSE by users who aren’t you (to get round the Amazon equivalent of Barbie doll and My Little Pony “items for you” recommendations I seem to get after every Christmas!), but exposing the personal search engine would potentially be a way of exposing a valuable commodity.

In the context of the Library, rather going to the Library website and looking up the books by a particular author, or going to ORO and looking up a particular author’s papers, you might pull their personalised search engine off the shelf and use that for a bit of a topic related Googling…

In a comment to Martin’s post, Laura suggests “Aren’t the search results that the expert actually follows up and bookmarks more powerful? Every academic should be publishing the RSS feeds for their social bookmarks, classified by key terms. The user can choose to filter these according to the social rating of the URL and aggregate results from a group of experts according to their reputation in their field and their online expertise in finding valuable sources.”

I guess this amounts to a social variant of the various “deliSearch” search engines out there, that let you run a search over a set of bookmarked pages or domains (see Search Hubs and Custom Search at ILI2007, for example, or these random OUseful posts on delicious powered search etc)?

At times like these, I sometimes wish I’d put a little more effort into searchfeedr (example: searching some of my delicious bookmarks tagged “search’ for items on “personal search”). I stopped working on searchfeedr before the Google CSE really got going, so it’d be possible to build a far more powerful version of it now…

Anyway, that’s maybe something else to put on the “proof-of-concept to do” list…

PS Note to self – also look at “How Do I?” instructional video search engine to see how easy it would be to embed videos in the results…

Getting an RSS Feed Out of a Google Custom Search Engine (CSE)

Alan posted me a tweet earlier today asking me to prove my “genius” credentials (heh, heh;-):

As far as I know, Google CSEs don’t offer an RSS output (yet: Google websearch doesn’t either, though rumour has it that it will, soon… so maybe CSEs will open up with opensearch too?)

So here’s a workaround…

If you make a query in a Google CSE – such as the rather wonderful How Do I? instructional video CSE ;-) – you’ll notice in the URL an argument that says &cx=somEGobbleDYGookNumber234sTUfF&cof….

google cse

The characters between cx= and either the end of the URL or an ampersand (&) are the ID of the CSE. In the case of How Do I?, the ID is 009190243792682903990%3Aqppoopa3lxa – almost; the “%3A” is a safe encoding for the web of the character “:”, so the actual CSE ID is 009190243792682903990:qppoopa3lxa. But we can work round that, and work with the encoded CSE ID cut straight from the URL.

Using the Google AJAX search API, you can create a query on any CSE that will return a result using the JSON format (a javascript object that can be loaded into a web page). The Google AJAX search API documentation tells you how: construct a Google AJAX web search query using the root http://ajax.googleapis.com/ajax/services/search/web?v=1.0 and add a few extra arguments to pull in results from a particular CSE: Web Search Specific Arguments.

JSON isn’t RSS, but we can get it into RSS quite easily, using a Yahoo pipe…

Just paste in the ID of a CSE (or the whole results URL), add your query, and subscribe to the results as an RSS feed from the More Options menu:

The pipe works as follows…

First up, create a text box to let a user enter a CSE ID cut and pasted from a CSE results page URL (this should work if you paste in the whole of the URL of the results page from a query made on your CSE):

Then create the search query input box, and along with the CSE ID use it to create a URL that calls the Google AJAX API:

Grab the JSON data feed from the Google AJAX Search API and translate the results so that the pipe will output a valid RSS feed:

And there you have it – an RSS feed for a particular query made on a particular Google CSE can be obtained from the Get as RSS output on the pipe’s More Options menu.

Time to Get Scared, People?

Last week, I posted a couple of tweets (via http://twitter.com/psychemedia) that were essentially doodles around the edge of what services like Google can work out about you from your online activity.

As ever in these matters, I picked on AJCann in the tweets, partly because he evangelises social web tool use to his students;-)

So what did I look at?

  • the Google Social Graph API – a service that tries to mine your social connections from public ‘friendships’ on the web. Check out the demo services…

    For example, here’s what the Google social API can find from Alan’s Friendfeed account using the “My Connections” demo:

    • people he links to on twitter and flickr;
    • people who link to him as a contact on twitter, delicious, friendfeed and flickr;
    • a link picked up from Science of the Invisible (which happens to be one of Alan’s blogs), also picks out his identi.ca identity; adding that URL to the Social Graph API form pulls out more contacts – via foaf records – from Alan’s identi.ca profile;

    The “Site Connections” demo pulls out all sorts of info about an individual by looking at URLs prominently associated with them, such as a personal blog:

    The possible connections reveal Alan’s possible identity on Technorati, Twitter, identi.ca, friendfeed, swurl, seesmic and mybloglog.

  • For anyone who doesn’t know what Alan looks like, you can always do a “face” search on Google images;
  • increasingly, there are “people” search engines out there that are built solely for searching for people. One example is Spock (other examples include pipl, zoominfo and wink [and 123people, which offers and interesting federated search results page]). The Spock “deep web” search turns up links that potentially point to Alan’s friendfeed and twitter pages, his revver videos, slideshare account and so on;
  • Alan seems to be pretty consistent in the username he uses on different sites. This makes it easy to guess his account on different sites, of course – or use a service like User Name Check to do a quick search;

Now I wasn’t going to post anything about this, but today I saw the following on Google Blogoscoped: Search Google Profiles, which describes a new Google search feature. (Didn’t know you had a Google Profile? If you have a Google account, you probably do – http://www.google.com/s2/profiles/me/? And if you want to really scare yourself with what your Google account can do to you, check http://www.google.com/history/… go on, I dare you…)

I had a quick look to see if I could find a link for the new profile search on my profile page, but didn’t spot one, although it’s easy enough to find the search form here: http://www.google.com/s2/profiles. (Maybe I don’t get a link because my profile isn’t public?)

Anyway, while looking over my profile, I thought I’d add my blog URL (http://ouseful.info) to it – and as soon as I clicked enter, got this:

A set of links that I might want to add to my profile – taken in part from the Social Graph API, maybe? Over the next 6 months I could see Google providing a de facto social network aggregation site, just from re-posting to you what they know about your social connections from mining the data they’ve crawled, and linking some of it together…

And given that the Goog can learn a lot about you by virtue of crawling public pages that are already out there, how much more comprehensive will your profile on Google be (and how certain will it be in the profile it can automatically generate around you?) if you actually feed it yourself? (Bear in mind things like health care records exist already…)

PS I just had a look at my own Web History page on Google, and it seems like they’ve recently added some new features, such as “popular searches related to my searches”, and also something on search trends that I don’t fully (or even partially) understand? Or maybe they were already there and I’ve not noticed before/forgotten (I rarely look at my search history…)

PPS does the web know when your birthday is??? Bewar of “Happy Birthday me…”. See also My Web Birthday.

[Have you heard about Google’s ‘social circle’ technology yet? read more]