Archive for the ‘Analytics’ Category
A quick peek at the quick-off-the-mark users of the altc2011 hashtag on Twitter…
Social connections between folk using the hashtag:
(Image generated using gephi; node size: betweenness centrality, colour – follower count)
By looking at the Twitter profile of hashtag users, finding a user’s blog (or other affiliation) URL, and running RSS feed autodiscovery over the URLs, we can generate an OPML blogroll (after a fashion) from the list of hashtagging twitter users: altc2011 hashtaggers – discovered feeds OPML blogroll
List intelligence: I looked at the lists that hashtag users are on and ranked lists by number of subscribers as well as number of hashtag users appearing on the lists.
Lists containing N numbers of people using the altc2011 hashtag:
Lists ordered by subscriber count (first number is number of people on list who’ve been an early user of altc2011 hashtag):
/kamyousaf/e-learning-uk 27 107
/kamyousaf/uk-ict-education 14 80
/mhawksey/purposed 24 42
/mhawksey/lak11 20 34
/helenwhd/e-learning 43 31
/suebecks/tech-enhanced-learning 27 27
/catherinecronin/education-elearning 17 26
/amcunningham/learning 17 26
/juliadesigns/education-uk-18 21 25
/JonPowles/education 26 19
/PatParslow/elearning-crew 15 18
/mhawksey/jiscel10 19 14
/ousefulAPI/altc2010 52 12
/ZoeEBreen/elearning-evangelists-uk 20 9
/ulcc/mootuk11-taggers 18 9
/HeyWayne/learning-tech-people 15 9
If we look at membership of lists containing altc2011 members, and then see who appears on those lists, we get an idea (maybe) of notable people in the community (number is number of lists each person appeared on):
As a long time fan of custom search engine offerings, I keep wondering why Google doesn’t seem to have much active interest in this area? Google Custom Search updates are few and far between, and typically go unreported by the tech blogs. Perhaps more surprisingly, Custom Search Engines don’t appear to have much, if any, recognition in the Google Apps for Education suite, although I think they are available with a Google Apps for education ID?
One of the things I’ve been mulling over for years is the role that automatically created course related search engines might have to play as part of a course’s VLE offering. The search engine would offer search results either over a set of web domains linked to from the actual course materials, or simply boost results from those domains in the context of a “normal” set of search results. I’ve recently started thinking that we could also make use “promoted” results to highlight specific required or recommended readings when a particular topic is searched for (for example, Integrating Course Related Search and Bookmarking?).
During an informal “technical” meeting around three JISC funded reseource discovery projects at Cambridge yesterday (Comet, Jerome, SALDA; disclaimer: I didn’t work on any of them, but I was in the area over the weekend…), there were a few brief mentions of how various university libraries were opening up their catalogues to the search engine crawlers. So for example, if you do a site: limited search on the following paths:
you can get (partial?) search results, with a greater or lesser degree of success, from the Sussex, Lincoln, Huddersfield and Cambridge catalogues respectively.
In a Google custom search engine context, we can tunnel in a little deeper in an attempt to returns results limited to actual records:
I’m not sure how useful or interesting this is at the moment, except to the library systems developers maybe, who can compare how informatively their library catalogue content is indexed and displayed in Google search results compared to other libraries… (so for example, I noticed that Google appears to be indexing the “related items” that Huddersfield publishes on a record page, meaning that if a search term appears in a related work, you might get a record that at first glance appears to have little to do with your search term, in effect providing a “reverse related work” search (that is, search on related works and return items that have the search term as the related work)).
But it’s a start… and with the addition of customised rankings, might provide a jumping off point for experimenting with novel ways of searching across UK HE catalogues using Google indexed content. (For example, a version of the CSE on the cam.ac.uk domain might boost the Cambridge results; within an institution, works related to a particular course through mention on a reading list might get a boost if a student on that course runs a search… and so on…
PS A couple of other things that may be worth pondering… could Google Apps for Education account holders be signed up to to Subscribed Links offering customised search results in the main Google domain relating to a particular course. (That is, define subscribed link profiles for a each course, and automatically add those subscriptions to an Apps for Edu user’s account based on the courses they’re taking?) Or I wonder if it would be possible to associate subscribed links to public access browsers in some way?
And how about finding some way of working with Google to open up “professional” search profiles, where for example students are provided with “read only” versions of the personalised search results of an expert in a particular area who has tuned, through personalisation, a search profile that is highly specialised in a particular subject area, e.g. as mentioned in Google Personal Custom Search Engines? (see also Could Librarians Be Influential Friends? And Who Owns Your Search Persona?).
If anyone out there is working on ways of using Google customised and personalised search as a way of delivering “improved” search results in an educational context, I’d love to hear more about what you’re getting up to…
Just so I don’t forget the development timeline such as it is, here are a few quick notes-to-self as much as anything about my “List Intelligence” tinkering to date:
- List Intelligence uses (currently) Twitter lists to associate individuals with a particular topic area (the focus of the list; note that this may be ill-specified, e.g. “people I have met”, or topic focussed “OU employees”, etc)
- List Intelligence is presented with a set of “candidate members” and then:
- looks up the lists those candidate members are on to provide a set of “candidate lists”;
- identifies the membership of those candidate lists (“candidate list members”) (this set may be subject to ranking or filtering, for example based on the number of list subscribers, or the number of original candidate members who are members of the current list);
- for the superset of members across lists (i.e. the set of candidate list members), rank each individual compared to the number of lists they are on (this may be optionally weighted by the number of subscribers to each list they are on); these individuals are potentially “key” players in the subject area defined by the lists that the original candidate members are members of;
- identify which of the candidate lists contains most candidate members, and rank accordingly (possibly also according to subscriber numbers); the top ranked lists are lists trivially associated with the set of original candidate members;
- provide output files that allow the graphing of individuals who are co-members of the same sets, and use the corresponding network as the basis for network analysis;
- optionally generate graphs based on friendship connections between candidate list members, and use the resulting graph as the basis for network analysis. (Any clusters/communities detected based on friendship may then be compared with the co-membership graphs to see the extent to which list memberships reflect or correlate to community structures);
- the original set of candidate members may be defined in a variety of ways. For example:
- one or more named individuals;
- the friends of a named individual;
- the recent users of a particular hashtag;
- the recent users of a particular searched for term;
- the members of a “seed” list.
- List Intelligence attempts to identify “list clusters” in the candidate lists set by detecting significant overlaps in membership between different candidate lists.
- Candidate lists may be used to identify potential “focus of interest” areas associated with the original set of candidate members.
I’ll try to post some pseudo-code, flow charts and formal algorithms to describe the above… but it may take a week or two…
Assuming my projects haven’t been cut out at the final acceptance stage because I haven’t yet submitted a revised project plan,
As OU courses are increasingly presented through the VLE, many of them opt to have one or more “Library Resources” pages that contain links to course related resources either hosted on the OU Library website or made available through a Library operated web service. Links to Library hosted or moderated resources may also appear inline in course content on the VLE. However, at the current time, it is difficult to get much idea about the extent to which any of these resources are ever accessed, or how students on a course make use of other Library resources.
With the state of the collection and reporting of activity data from the VLE still evolving, this project will explore the extent to which we can make use of data I do know exists, and to which I do have access, specifically Google Analytics data for the library.open.ac.uk domain.
The intention is to produce a three-way reporting framework using Google Analytics for visitors to the OU Library website and Library managed resources from the VLE. The reports will be targeted at: subject librarians who liaise with course teams; course teams; subscription managers.
Google Analytics (to which I have access) are already running on the library website and the matter just(?!) arises now of:
1) Identifying appropriate filters and segments to capture visits from different courses;
2) development of Google Analytics API wrapper calls to capture data by course or resource based segments and enable analysis, visualisation and reporting not supported within the Google Analytics environment.
3) Providing a meaningful reporting format for the three audience types. (note: we might also explore whether a view over the activity data may be appropriate for presenting back to students on a course.)
The OU Library has been running Google Analytics for several year, but to my knowledge has not started to exploit the data being collected as part of a reporting strategy on the usage of library resources resulting from referrals from the VLE. (Whenever a user clicks on a link in the VLE that leads to the Library website, the Google Analytics on the Library website can capture that fact.)
At the moment, we do not tend to work on optimising our online courses as websites so that they deliver the sorts of behaviour we want to encourage. If we were a web company, we would regularly analyse user behaviour on our course websites and modify them as a result.
This project represents the first step in a web analytics approach to understanding how our students access Library resources from the VLE: reporting. The project will then provide the basis for a follow on project that can look at how we can take insight from those reports and make them actionable, for example in the redesign of the way links to library resources are presented or used in the VLE, or how visitors from the VLE are handled when they hit the Library website.
The project complements work that has just started in the Library on a JISC funded project to making journal recommendations to students based on previous user actions.
The first outcome will be a set of Google Analytics filters and advanced segments tuned to the VLE visitor traffic and resource usage on the Library website. The second will be a set of Google analytics API wrappers that allow us to export this data and use it outside the Google Analytics environment.
The final deliverables are three report types in two possible flavours:
1) a report to subject librarians about the usage of library resources from visitors referred from the VLE for courses they look after
2) a report to librarians responsible for particular subscription databases showing how that resource is accessed by visitors referred from the VLE, broken down by course
3) a report to course teams showing how library resources linked to from the VLE for their course are used by visitors referred to those resources from the VLE.
The two flavours are:
a) Google analytics reports
b) custom dashboard with data accessed via the Google Analytics API
Recommendations will also be made based on the extent to which Library website usage by anonymous students on particular OU courses may be tracked by other means, such as affinity strings in the SAMS cookie, and the benefits that may accrue from this more comprehensive form of tracking.
If course team members on any OU courses presenting over the next 9 months are interested in how students are using the library website following a referral from the VLE, please get in touch. If academics on courses outside the OU would like to discuss the use of Google Analytics in an educational context, I’d love to hear from you too:-)
eSTEeM is joint initiative between the Open University’s Faculty of Science and Faculty of Maths, Computing and Technology to develop new approaches to teaching and learning both within existing and new programmes.