digital scholarship – OUseful.Info, the blog…

A Question About Klout…

I’ve no idea how Klout works out it’s scores, but I’m guessing that there is an element of PageRank style algorithmic bootstrapping going on, in which a person’s Klout score is influenced by the Klout score of folk who interact with a person.

So for example, if we look at @briankelly, we see how he influences other influential (or not) folk on Klout:

One thing I’ve noticed about my Klout scrore is that it tends to be lower than most of the folk I have an OU/edtech style relationship with; and no, I don’t obsess about it… I just occasionally refer to it when Klout is in the news, as it was today with an announced tie up with Bing: Bing and Klout Partner to Strengthen Social Search and Online Influence. In this case, if my search results are going to be influenced by Bing, I want to understand what effect that might have on the search results I’m presented with, and how my content/contributions might be being weighted in other peoples’ search results.

So here’s a look at the Klout scrores of the folk I’ve influenced on Klout:

Hmm… seems like many of them are sensible and are completely ignoring Klout. So I’m wondering: is my Klout score depressed relative to other ed-tech folk who are on Klout because I’m not interacting with folk who are playing the Klout game? Which is to say: if you are generating ranking scores based at least in part on the statistics of a particular netwrok, it can be handy to know what netwrok those stats are being measured on. If Klout stats are dominated by components based on networks statistics calculated from membership of the Klout network, that is very different to the sorts of scores you might get if the same stats were calculated over the whole of the Twitter network graph…

Sort of, but not quite, related: a few articles on sampling error and sample bias – Is Your Survey Data Lying to You? and The Most Dangerous Porfession: A Note on Nonsampling Error.

PS Hmmm.. I wonder how my Technorati ranking is doing today…;-)

Scholarly Communication in the Networked Age

Last week, I was fortunate enough to receive an invitation to attend the Texts and Literacy in the Digital Age: Assessing the future of scholarly communication at the Dutch National Library in Den Haag (a trip that ended up turning into a weekend break in Amsterdam when my flight was cancelled…)

The presentation can be found here and embedded below, if your feed reader supports it:

One thing I have tried to do is annotate each slide with a short piece of discursive text relating to the slide. I need to find a way of linearising slide shows prepared this way to see if I can find a way of generating blog posts from them, which is a task for next year…

The presentation draws heavily on Martin Belam’s news:rewired presentation from 2009 (The tyranny of chronology), as I try to tease out some of the structural issues that face the presentation of news media in an online networked age, and constrast (or complement) them with issues faced by scholoarly publishing.

One of the things I hope to mull over more next year, and maybe communicate in a more principled way rather than via occasional blog posts and tweets, are the ways in which news media and academia can work together to put the news into some sort of deeper context, and maybe even into a learning (resource) context…

Unmeasurable Impact

~~Lots of deleted stuff I might have regretted posting…~~

(I also apologise in advance for what some might take to be the self-aggrandising nature of this post…)

Anyway, that’s all as maybe… One of the ideas I started trying to develop in preparing the promotion case was the notion of “influence”, and how online, network based activities might result in payoff for someone else, through being influenced, that could in part trickle back through some sort of recognised acknowledgement, or feed forward into a payoff that makes the academic or host institution more productive.

So here are a handful of examples from the last week or so that provide anecdotal evidence about the influence and reach of posts appearing on OUseful.info:

I flashed up on screen a post from Tony Hirst’s OUseful blog where he confessed to ‘hassling’ Simon Rogers over the formats of some of the information in the Guardian Datastore.

Tony’s contributions are fantastically useful, and the team have now changed some of their workflows to try and include more universal identifiers. On datasets with country lists, for example, they now aim to provide the two letter ISO country code in order to get around confusion when comparing datasets that might feature Burma or Myanmar for example.

– news:rewired – “Reader-centred journalism”

[C]hanged some of their workflows… right… so that might make it easier for others, such as academics stooping so low as to use news media published data rather than “original” sources in their own work. Or it might mean that folk who are not academics putting the data to work because it’s now easier for them to do so, and getting real value out of it.

(Academic bashing? Me? Surely not… Though of course, I have come to realise over the last year that I am absolutely not considered an academic by the academy…)

Here’s the second example, referring to some “work” that resulted from an open exchange over a weekend earlier this year which Cameron Neylon reviewed in A little bit of federated Open Notebook Science. The context is graphing user and compound interactions by extracting the appropriate bipartite graph from a set of open notebooks:

We are very fortunate that Don Pellegrino, an IST student at Drexel, has selected the analysis of networks within Open Notebooks as part of his Ph.D. work. He has started to report his progress on our wiki and is eager to receive any feedback as the work progresses (his FriendFeed account is donpellegrino).

Don’s first report is available here. He is using the Open Source software Gephi for visualization and has provided all of the data and code on the associated wiki page. (also see Tony Hirst’s description of mapping ONS work which provided some very useful insights) Don has provided a detailed report of his findings but I think the most important can be seen in the global plot below.

– Visualizing Social Networks in Open Notebooks

[S]ome very useful insights – right, a couple of approximately and quickly worked through examples that sketched out some possible ways of looking at this area, as well as crude proof of concept demos; it maybe also identified some dead-ends that might otherwise have been pursued?

Finally, this from Brian Kelly:

Niall Sclater made his point succinctly:

@mweller @psychemedia delicious. i rest my case.

The case Niall was making was, I suspect, that one shouldn’t be promoting use of Cloud services within institutions. This is an argument (although that might be putting it a bit too strongly) which Niall has been having over the past few years with Tony Hirst and Martin Weller, his colleagues at the Open University. As I described in a post on “When Two Tribes Go To War” back in 2007:

Niall Sclater, Director of the OU VLE Programme at the Open University recently pointed out that the Slideshare service was down, using this as an “attempt to inject some reality into the VLEs v Small Pieces debate“. His colleague at the Open University, Tony Hirst responded with a post entitled “An error has occurred whilst accessing this site” in which Tony, with “beautifully sweet irony“, alerted Niall to the fact that the OU’s Intranet was also down.

Back then the specifics related to the reliability of the Slideshare service, with Tony pointing out the the Slideshare service was actually more reliable that the Open University’s Intranet. But that was just a minor detail. The leaked news that Yahoo was, it appeared, intending to close a social bookmarking services which is highly regarded by many of its users, was clearly of much more significance. So is Niall correct to rest his case on this news? Or, as Niall wrote his tweet before we found that the news of Delicious’s death was greatly exaggerated, might we feel that the issue is now simply whether an alternative social bookmarking service should be used?

– Lessons From Delicious’s (Non)-Demise

What this example shows, and maybe the one before it too, is that the very act of working in open and in public means that the process of the work/interaction as well as the “work” itself can become the focus of (authentic) stories in other people’s work. Brian has been telling the above story repeatedly over the last few years, which has the side-effect of raising the OU’s profile as an institution that is *really* engaged with these issues.

None of the above anecdotes has resulted in an academic citation for me, so none of it counts in academic terms. None of the above resulted in the OU being paid for the time I spent engaged in the related activities, so it none if it helped the OU bottom line directly (we’re really, really a business now, right?). None of the above ended up in any OU course materials (to my knowledge). It was all, from my perspective looking round my current institutional role, pointless…

PS it’s worth noting that, through trackbacks and email requests, I see these ephemeral “been influenced by” signals on my web radar as a matter of course. But my internal profile is largely below the radar, and these “influence signals” are likely to be even more invisible. This maybe suggests that my reach is only to folk who look outwards (from any institution), using the web, or the people who see me give a presentation (which I do once a month or so)… Hmm…

Should Academic Journal Papers Have Video Trailers?

I don’t read academic journal papers very much any more, partly because folk rarely link to them, but today I read a paper (“Narrative Visualization: Telling Stories with Data”, Edward Segel, Jeffrey Heer, IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis), 2010) in response to this video trail that brought it to my attention (Journalism in the Age of Data, Ch. 3: Telling “Data Stories”):

I encourage you to watch the video – not necessarily for what it’s about, but for the way that a journal article is used to hold bits of the video together. Note that the video is not just about the paper, but it’s not hard to see how a video could be made that was just about the paper…

So I wonder: should we be making voiced over “papercasts” of academic papers to provide a quick summary of what they contain, and maybe also enriching them with photos and footage relating to what the content of the paper is about? (I know this might not make sense for the subject matter of every paper, but if a journal paper is about a particular online tool, for example, here would be an opportunity to show a few seconds of the tool in use, and contextualise it/demonstrate it a little more interestingly than a single, simple screenshot can convey?)

UPDATE: @der_no tweets: “Always enjoyed technical papers preview @ #SIGGRAPH (esp considering many of actual papers are beyond me)” See an example conference papers trailer here – SIGGRAPH 2010 : Technical Papers Trailer:

If the conference matter is appropriate (robotics related conferences come to my mind, for example), couldn’t this sort of approach provide an additional legacy resource that can continue to give an event life after the fact?

PS I believe that several of the OpenLearn folk are also looking at ways of pulling together video and audio in the way they package their material, for example looking at the use of Xtranormal videos, or Slideshare slidecasts. (Note that it’s easy (or used to be!) to publish Xtranormal clips into Youtube, and Youtube clips can also be embedded in Slideshare presentations, so all manner of fusions of content become possible!)

PPS Very, very loosely related to the above is another thread I want to link in to, here. That is, the extent to which academics might take up various sorts of (“new”) media training to explore different ways of engaging with (and maybe helping reinvent?) scientific communication. For example, a recent initiative in the OU has seen more than a few brave academic volunteers engaging in podcast training as part of Martin’s Podstars project (I couldn’t find a better link?!).

Running parallel to this, the OBU’s media training team have been helping other academics put together short showreels that have since been published on the OU podcast site – OU Experts:

In terms of finding training materials that are already out there, it struck me that the BBC College of Journalism might be a good start, particularly in the skills area?

So WTF is Data Scholarship?

A tweet just passed me by from @andypowell at today’s Linked Data: The Future of Knowledge Organization on the Web event:

“need to introduce data literacy into education in order to create data literate citizens” closing remarks by nigel shadbolt at #isko

With the OU’s infoskills short course Beyond Google: working with information online in it’s last week of registration for its last presentation, it may be that there’ll be a slot open in the short course programme in a year or two for an OU course on data literacy (and visualisation…?!;-), but in the meantime, to justify some of things I’m getting up to, I suspect I’m going to have to try to persuade folk that there’s some merit in figuring out what sorts of tools make sense in the world of unlimited open data and data scholarship…

Now I have to admit that I’m not sure at all sure what data scholarship is, or might be (same with data literacy… sigh…) but here are a few possible starters for ten…

I first came across something close to the phrase whilst at the Repository Fringe, searching for papers relating to referencing data, in a preprint from Peter Murray Rust – Open Data in Science: “Recent initiatives such as the JISC/NSF report on cyberscholarship have emphasized the critical importance of data-driven scholarship.” Digging around the phrase turned up one or two references to data citability as being a key requirement for data(-driven) scholarship, a point also touched on by Kevin Ashley in his closing keynote at the RepoFringe. In particular Kevin referenced Peter Buneman‘s work on that very topic, which in a roundabout way led me to finding a paper by Bruce Barkstorm on Digital libraries and data scholarship, which again looks at some of the issues involved in referencing data. (I’ll do a post or two on data referencing – something I need to improve in my own practice – at some point…)

So for example, the abstract to Barkstrom’s paper begins: “In addition to preserving and retrieving digital information, digital libraries need to allow data scholars to create post-publication references to objects within files and across collections of files” before going on to discuss referencing matters. So implicitly, data scholarship must be something to do with poring through other peoples’ old data…

I’m still not sure I know what a data scholar might actually do though, or why, although it seemingly requires ability to reference data, so I took a sideways step to review what a digital scholar might be… Martin Weller has posted about this previously (e.g. Thoughts on digital scholarship), relating the idea to Boyer’s notions of scholarship (discovery, integration, application, teaching).

A short unit on Connexions (What Is Digital Scholarship?) by the American Council of Learned Societies Commission on Cyberinfrastructure for the Humanities & Social Sciences suggests:

In recent practice, “digital scholarship” has meant several related things:

– Building a digital collection of information for further study and analysis*
– Creating appropriate tools for collection-building
– Creating appropriate tools for the analysis and study of collections
– Using digital collections and analytical tools to generate new intellectual products
– Creating authoring tools for these new intellectual products, either in traditional forms or in digital form

* like this 500 page bibliography on digital scholarship [via @jfj24]. My response: “is the idea that i read those 500 pages of citations and from titles alone form a coherent view about what’s involved?!” Heh heh ;-)

The piece goes on:

It may seem odd to some that creating collections and the tools to use them should be counted as scholarship, but humanities and social science research has always required collections of appropriate information, and throughout history, scholars have often been the ones to assemble those collections, as part of their scholarship. Moreover, scholars have been building tools since the first index, the first concordance, the first scholarly edition. So, while it is reasonable to regard (d) as the core meaning and ultimate objective of “digital scholarship,” it is also important to recognize that in the early digital era, leadership may well consist of collection-building or tool-building. In addition, tool-building is dependent on the existence of collections, and both collections and tools get better and more general as there is more use of digital information. If we hope to see new intellectual products, we should give high priority to building tools and collections. Finally, it is worth noting that although (a), (b), (c), and (e) require a great deal of cooperation, it is still imaginable that (d) can be the work of a single individual.

Remember, I am in part looking for a definition of data scholarship to justify spending time on OUseful things, so maybe here we have something like it…? Because I think I can argue that OUseful.info identifies/discovers useful tools, integrates them within an information processing context that includes other tools and services, applies them to particular “real world” examples, and then teaches on (sort of!) how to do the same (so that’s Boyer’s boxes ticked). In addition, some of the integrations I come up with could be classed as the development of new tools in their own right, and as far as collections go: I’ve always been keen on trying to make “discovered context” tangible, as with the discovered search engines I’ve blogged about recently.

PS quickly skimming the above, it seems to me that scholarship maybe has a couple of facets: firstly, the development and identification of tools and techniques that allow “scholars” to do what they do; secondly, the use of those tools and techniques to make sense and meaning of things produced by others beyond the sense and meaning that they themselves have extracted. Recalling the idea that the most interesting thing that will be done with your data will be done by someone else, maybe that’s what scholars do?

Google Impact…? The “Google Suggest” Factor

So apparently, OU promotion process means I get feedback on my promotion case before it goes to the full committee… Here’s what I need to address:

– put references in proper OU CV style;
– don’t write career history or list stuff done in the promotion case, instead list impact and significant contributions; [but for a case based around digital engagement, that can be hard to judge…? I wonder whether the ability to drive traffic to the OU would count? I wonder if I could create a traffic blip on an OU web page anyway? If you fancy taking part in an ad hoc, not really experiment to see if I can drive traffic to the OU, please click through here: OU Accreditations and Partnerships…)

What other sorts of impact are there? Eponymous laws? Google impact… Hmmm…. How about a Google Suggest factor…?

Let’s see… (whilst I made this searches in a browser that wasn’t logged in to Google, and had cookies cleaned, I’m not suggesting any Google ground truth in these “results”.)

(the actor/voiceover artist isn’t me… ;-)

So what ingredients might go into a “Google Suggest” Impact Factor?

Number of correct mentions? Number of incorrect mentions? Explicit association with host university, or subject area?

And what might a Google Suggest Factor measure? Personal discoverability? Personal associations? Personal specialism areas?

One thing I didn’t manage to do was find any phrases that autosuggested a name from a term in the following way:

i.e. term firstname surname

So what does Google Suggest think about you?! ;-)

Related, in a roundabout sort of way: Where is the Open University Homepage?

Brand Association and Your Twitter Followers

One of the thing’s that Martin appears to have been thinking a lot about lately are metrics for rating ‘digital scholars’, i.e. those of us who don’t do any of the reputation bearing thing that traditional academics do, (though whether that’s because we’re not very good at those things is not for me to say;-)

So for example, in The Keynote Equivalent? he reviews the notions of reputation and impact factor, and in Connectors versus outputs he calls on some really dubious social media buzz metrics to raise the more far more valid issue of how we measure influence within, and value the contributions made to, a social network (peer community?) in order to recognise the extent of someone’s influence within that network from outside of it.

Using Twitter as a base case, one of the many interesting features of Twitter’s ‘open privacy’ model is that in most cases it’s possible for you to look at someone else’s followers to see who they are.

The value of that network to an individual is at least twofold – firstly, as a source of information, observations, news and feedback to you as the person at the centre of your own network; secondly as an amplifier of your own ~~ego~~ broadcast messages. (There are other benefits of course – like being able to see who is talking to whom about what.) You may also feel there is some benefit to just having a large number of followers, if only in the bragging stakes.

That is, the more followers the better, right? It’s bound to be good for my reputation, if nothing else, surely…?

Well….. maybe not…?

Consider these two questions:

– who follows you? if I look at your followers what can I tell about you, from them?
– what is your blocking policy? who you block is just as much a part of the way you manage your network as the people you actively follow.

As far as my own Twitter network goes, I am on a follow:followed ratio of about 1:4. That is, approximately four times as many people follow me as I follow back. For every 10 or so new followers I get, I block one or two.

I check my followers list maybe once every two or three days, which lets me keep up with the pruning on just one or two screens of followers using the Twitter web interface. If the name or avatar is suspect, I’ll check out the tweets to see if I want to block. (I really miss the ability to hover over a person’s name and get a tooltip containing their bio:-( If the name or avatar is familiar or intriguing, I’ll check the tweets to see if I’m going to follow back (maybe 1 in 20? Following back is not the main source for me of new people to follow – you’ll have to get to me another way;-).

The people I block? People who’s tweets are never replies, but who just tweet out advertising links all the time; Britney, whatever she happens to be sucking or loving at the time; product tweeters; and so on. If you’re following lots of people and only followed by a few? Not good – why should I follow you if no-one else does? If you’re following lots of people and are followed by lots of people? Also not good: either you’re a spammer being spammed back, or you’re an indiscriminate symmetric follower so why should I trust you, or you’ve so many followers I’m not going to get a look in. If I’m not sure about a new follower, it’s 50/50 that I’ll either block them or not, so there may well be the odd false positive amongst the people I’ve blocked (if so, sorry…) And why do I block them? Because they add no value to me… Like junk mail… And because by association, if you look at my followers and see they’re all Britney, you’ll know my amplification network is worthless. And by association… ;-)

The people I follow? People I’ve chatted to, have been introduced to through RTs, or via interesting/valuable multiaddressed tweets that include me; people who appear not to be part of any other network I follow (or who might add value in a sphere of influence or interest that I don’t feel I currently benefit from), and so on.

And the people I don’t follow but don’t block (i.e. the majority) – nothing personal, but I only have so many hours in the day, and can’t cope with too many new messages every update cycle in by twitter client!

So all this might sound a little bit arrogant, but it’s my space and it’s me that has to navigate it!

PS just by the by, it struck me during an exchange last week that networks can also act as PR channels. A tweet went out from @ruskin147 asking if anyone knew anyone “who can analyse how viral emails,campaigns etc, can knock a firm off course?” Now I should probably have recommended someone from the OU Business School, because I think there is someone there who knows this stuff; but they’re not part of any of my networks so I’d have to go and search for them and essentially recommend them cold. So instead I suggested @mediaczar (who blogs under the same ID) because he’s been sharing code and insight about his analysis of connectivity and the flow of ideas across social networks for the PR firm (I think?) he works for. (Some irony there, methinks?;-) And it turned out that the two of them hooked up and had a chat…

So why’s that good for me? Because it strengthened the network that I inhabit. It increased the likelihood of those people having an interesting conversation that I was likely to also be interested in. I get value not just from people telling me things, but also from people in my network telling each other things that I am likely to find interesting.

And as a spin-off, it maybe increases my reputation with those two people for having helped create that conversation between them?

In terms of externally recognised value though? How are you going to measure that, Martin?