OUseful.Info, the blog…

Trying to find useful things to do with emerging technologies in open education

Deriving a Persistent EdTech Context from the ALTC2010 Twitter Backchannel

So you’ve been to an event where everyone was tweeting, and now what? That stuff’s all in the past, right? Wrong…

Earlier today, I published a short post describing how it was possible to do all sorts of wonderful things around a twitter hashtag community (well I think they’re wonderful – or some of them, at least…). In this post, I’ll give a couple of illustrations using the #altc2010 hashtag from this year’s ALTC conference.

First up, what does the inner structure of the hashtag community look like? That is, of the Twitter folk using the twitter hashtag (in fact, folk who’ve used the hashtag more than three times over the last couple of days), who follows whom? In the following graph, nodes are individual twitterers, edges go from a person to a person they follow, node size and label size is proportional to the number of hashtaggers following the named person (that is, the in degree of the node) and colour is proportional to the number of hashtaggers an individual is following (out degree; red is “hot”/high).

ALTC-2010 hashtag community

This graph was produced using Gephi, which can also run stats over the graph. So for example, if we size the nodes according to betweenness, we can see which twitterers in the community are likely to be most effective at getting a message out across that community.

ALC2010 BEtweenness centrality

Note that the ALT user is way and above the node with the highest betweenness score – the sizes of the other nodes are amplified just so we can see them…

If we grab the total number of followers and friends of each user (that is, including folk who have not used the hashtag and are not part of the hashtag community) and use that to set the size (number of followers) and colour (number of friends) of each user, we can see which twitterers are most likely to amplify the event outside of the community.

ALTC2010 total frinds/followers

Okay, so what else can we do?

One thing is create a twitter list containing the folk who’ve been using the ALTC2010 hashtag; you can find it here: ALTC2010 List

ALTC2010 hashtaggers list

We can also feed the address of this list into a Yahoo pipe (described here) that will search through recent tweets visible through the list for hashtags. In this way we can use the folk who were twittering around ALTC2010 to act as an early warning beacon for other hashtags or hashtagged events in the educational technology area.

ALTC2010 hashtag community - what else is hot?

Something else we can do via the twitter list is grab everyone’s personal homepage URL, as declared on their twitter profile, and use these URLs to seed an ALTC2010 custom search engine.

ALTC2010 hashtaggers search engine

That is, a search engine over a good proportion of the personal pages of HE related UK educational technologists, as they declared themselves over Twitter circa September 2010.

[UPDATE: and here's an example of why the community defined custom search engine might be interesting... via @eingang: Ouch! David White And The Dragon Slaying]

So, there we have it. The scripts are in place, so generating the screenshots, and writing this post, took waaaaaaaaaay longer than mining the twapperkeeper archive, setting up the lists, generating the graph files (though I still had to load them into gephi, lay them out and render them “by hand” i.e. by clicking a couple of buttons…) and seeding the custom search engine (which also had to be initially set up by hand).

But why bother? Well, my developing idea is that we can mine events to define (automatically) a context around a particular subject area or domain (for example, a set of people interested in an expert in the area), and then draw on this context for search and discovery at a later date (e.g. through monitoring their twitter feeds via an auto-generated list to see what they – as a group of independent individuals – are talking about severally together, or by searching over just their personal webpages).

PS odds on some f*****r has patented this approach; if they have, this was all my own work, and it was bleedin’ obvious, so s***w you, m**********r… sue me.

PPS idly mulling over what else I could do with the custom search engine, I seem to remember that it’s possible to tweak the ranking factors of results returned from particular sites in the CSE definition file… which means we could take things like the number of twitter followers, or the betweenness centrality of everyone within the hashtag community, and use this as a ranking factor? That is, we might use the twitter “reputation” of an individual, either in general terms (overall number of followers, say), or within a community (e.g. betweenness centrality) to boost or reduce the tanking of results returned from their pages within the custom search engine. And if anyone else out there thinks they have a patent on that idea, they can f**k right off too, cos I haven’t got the idea from you, either…

PPPS for a few immediate thoughts about where next with all of this, see Additional Thoughts on Generating a Persistent Context from an Event Tag

Written by Tony Hirst

September 8, 2010 at 8:19 pm

Posted in Tinkering, Visualisation

Tagged with ,

16 Responses

Subscribe to comments with RSS.

  1. Brilliant … as ever!

    Chris Jobling

    September 8, 2010 at 10:46 pm

  2. Beautifully done an enormously powerful! Keep up the good work.

    John Rigdon

    September 9, 2010 at 1:50 am

  3. Awesome! Humbled that I feature. :-)

    Doug Belshaw

    September 9, 2010 at 6:58 am

  4. [...] First thoughts on things that might be useful, one day… BlogAboutSearch « Deriving a Persistent EdTech Context from the ALTC2010 Twitter Backchannel [...]

  5. Interesting and insightful. Thanks for sharing.

    Ady

    September 9, 2010 at 8:15 am

  6. This is useful! Thanks for doing it and posting it.

    Cath

    September 9, 2010 at 10:42 am

  7. Awesome! Scripts, please? (I didn’t see these at http://blog.ouseful.info/2010/09/08/discovering-context-event-focusing/ either).

    Also, is there a way to auto-generate the Twitter list? Or do you do that part by hand?

    Jodi Schneider

    September 9, 2010 at 1:09 pm

    • A couple of the script fragments are available here: http://blog.ouseful.info/2010/08/25/doodlings-around-the-data-driven-journalism-round-table-event-hashtag-community/

      The list is autogenerated from a list of Twitter IDs via the Twitter API and pretty much reduces to the following using tweepy:

      for u in userlist:
      api.add_list_member(tag, u)

      where ‘tag’ is the list slug.

      Creating lists is really handy for doing other things around sets of Twitter users because when you pull data user names from the list, it comes alomg with the total number of friends and followers, the personal homepage URL from the user’s Twitter profile, and so on.

      I will posts scripts when I’m a little less rushed for time… (don’t want to let passwords and keys slip through;-)

      Tony Hirst

      September 9, 2010 at 2:20 pm

  8. Great stuff – thanks for thinking this through. We’re working on some related stuff over at US Dept of Ed. This will definitely figure for us and thanks for laying it out.

    Steve Midgley

    September 10, 2010 at 2:29 am

  9. Hi Tony

    Great stuff as usual. Going to try and do some of this for the CETIS conference in November.

    Sheila

    Sheila MacNeill

    September 10, 2010 at 9:30 am

  10. There would be a nice little business in ‘Your twitter conference solutions’ – or at least some nicely packaged and easy to use tools that do this automatically. Funding?
    Also interesting in what it shows about the blurred boundaries of conferences – some of those featured (me and you for starters) weren’t even there.

    Martin

    September 10, 2010 at 12:36 pm

  11. [...] of the conference – this builds on the cool aggregation of the twitter feeds from ALT-C that Tony Hirst has [...]

  12. [...] Tony also pointed out that most document stores have a structure comprised of how documents relate to each other, but we are not good at making use of that structure. He then demonstrated how Gephi can be used to visualise structures and data clusters across multiple data stores. This presents new ways of navigating the content and can be used to provide topic or facet based browsing on the cheap. Earlier this week Tony demonstrated exactly this kind of data visualisation by using Gephi, yahoo pipes and google custom search to analyse altc-2010 twitter streams. [...]

  13. [...] I’m on the topic of tweet analysis, Tony Hirst did some awesome work visualizing the tweets from #altc2010 and showing relationships between people, nodes, and the [...]

  14. [...] If you are interested in making your own custom searches for courses or events I recommend reading Alan Levine’s OPML to CSE recipe (OPML is one way to output the source url for a collection of RSS feeds) or Tony Hirst’s  work on hashtag communities. [...]

  15. [...] archiving it, or maybe mining it in order to generate “derived” or secondary products? Deriving a Persistent EdTech Context from the ALTC2010 Twitter Backchannel and Additional Thoughts on Generating a Persistent Context from an Event Tag – tag network [...]


Comments are closed.

Follow

Get every new post delivered to your Inbox.

Join 787 other followers

%d bloggers like this: