A couple of weeks ago I did a phone interview for the OU’s DISCO project – OU Digital Scholarship Portal. From what I remember of the call, it rambled over many and varied topics, including possible metrics that might be taken into account when putting together promotion cases that include a demonstration of excellence in digital scholarship (whatever that is…).
Anyway, today I wasted a day – a whole day – updating my CV and writing stuff that seems to be the wrong stuff for an OU promotion case. Ever the reflective sort(?!), here are some observations I came away with:
– Slideshare is my presentation memory; I need to get in the habit of recording the date and event a presentation is for when I upload it to make it easier to list the presentations I’ve given. Alternatively, it might make sense to use a calendar to record the dates and events I’ve spoken at and then use the iCal feed to display the result;
– not writing formal academic papers means I have nothing to cite that t’committee would accept as credible. However, I have given quite a few interviews over the last couple of years to folk writing formal reports, doing research projects, or writing books. I’ve also participated in a few Delphi exercises and attended invitation only workshops and brainstorming sessions, as well as being invited to speak at events folk pay money to attend. Here’s part of what I wrote on this topic in my draft case: I have all but given up on formal academic publishing, in favour of short-form informal blog posts, occasional articles, and interviews for people who are writing long-form pieces (books, reports) which typically offer a greater or more immediate reach than scholarly articles in refereed journals, or benefit from a greater impact or better targeted audience than I could personally reach. The problem? That whilst I regularly participate in interviews and conversations with people writing official reports, books, etc as well as participating in Delphi Exercises[,] I’m not very good at keeping records of these or tracking down citations…
What occurs to me, then, is that I am more interested in direct or immediate communications of ideas as part of an ongoing process of learning and discovery (as part of a conversation, to use that well-worn and increasingly pointless phrase…) rather than archiving ideas for the record. (This also reflects my cynical attitude that the majority of stuff that appears in the formal record is not, to my mind, a contribution to anything other than the bulk of a journal sold for profit…)
If I’m going into the archive, someone else can put me there… But for the promotion case, acknowledgements are the lowest of the low in terms of academic credibility, rivaled only by (pers comm). Which is a shame – because one of the quotes I carry with me (but unfortunately can’t credit because I can’t for the life of me remember who said it, except that it was someone from outside the OU giving a seminar in the OU), that the whole point of being an academic is to have interesting conversations.
Anyway, the reason why I started to write this post is this: if the digital scholarship folks want metrics around how effective a scholar’s online activities are, it may be worth looking at tangible outcomes in the real world – such as invitations (e.g. to speak at seminars and workshops) and acknowledgements (e.g. in books, articles and reports). This conversion from informal online activity to a formal request in physical space is where the “citation” is evidenced.
And as Stephen Downes writes in a recent Half an Hour post:
By sharing my work freely, people around the world are able to see it, and they willingly pay for me to come and speak to them. I do not collect speaker fees, but I do require that they pay my expenses, because otherwise I could not afford to travel to their cities. We both benefit, because I then use these trips to produce work that we share with other people around the world, and the cycle continues.
You might think, it’s not a very good deal for some organization to pay several thousand dollars to fly me to their city. But consider the cost were they to buy books from me instead. They could get maybe 30 or 40 copies of an academic text for the same amount. This way, they get all my content I ever create for free, as many copies as they would ever need. [Paying For Art]
If the point of publishing is to communicate ideas, then presentations count. And if the refereeing process is to guarantee quality, then being given an invitation to speak also reflects reputation brownie points and an element of trust on the part of the person responsible for extending the invitation, even if they are not explicitly evaluating the actual content of a presentation a priori.
As to the benefits accruing to Stephen’s employer: “[t]hey get the reputation from sponsoring my work” as well as influencing whatever he is working on.
I’m not sure what metrics Stephen uses if he goes through an annual staff development/appraisal cycle (I thought I’d read something he’d written on this before, but I can’t find it if he did…?) but it would be interesting to see them…
PS today has been crap day. The only enjoyable part has been this bit – thinking about how I might be able to build a living CV… Paraphrasing Fermat, if I didn’t have to walk the dog just now, I’d have been able to build the neatest little demonstration site for this, which would include parsing the events out of my CV into a spreadsheet, and then using my Maintaining a Google Calendar from a Google Spreadsheet recipe to get them into a calendar;-)
I had to do exactly the same thing myself late last year – trawling through emails and calendars to find invitations etc. (One tip: I did advanced searches for my name on foreign language sites which brought up examples of ‘international renown’ – I’m named on the Indonesian wikipedia!)
Thankfully, although I didn’t get the ‘result’ that I was applying for (not that I expected to), the applications from me and my colleague Andrew Dubber did prompt the university to set up a group to look at how they recognise reputation in a new media environment (like me, Dubber gets a lot of invitations to speak and has had huge traffic on his blog). The snail moves on…
I did my appraisal form this week, and decided to really list everything instead of just giving the main headlines. It was interesting going over my blog how much stuff there was that I’d forgotten. My blog is my memory for this now.
Agree about Slideshare – I put in four or five presentations and the number of views they’d had (2k-3k) as a measure of impact. If a paper had that many citations we’d be creaming ourselves, and they keep saying it’s impact that’s important.
The record vs conversation thing is interesting – you may remember I blogged about outputs vs connections a while back (http://nogoodreason.typepad.co.uk/no_good_reason/2009/06/connections-versus-outputs.html). Thing is committees and organisations like outputs – they can measure them. The shift from artifacts to network will take a long time I guess – and as you know people underestimate the amount of effort (or expertise) that is required to maintain or develop and be valued by a network
Good luck with it. For my tenure reviews they really only count peer-reviewed journal articles and student ratings (and grants), not book chapters, conference papers or presentations, student work, etc.
I like the idea of perhaps quantifying connections rather than just outputs, but that sounds like something they would lump under ‘service’, not research or teaching. And service is supposed to only be 10% of my effort.
Hi Tony, I thought you might find this link useful. Its from a humanities network, so a little verbose. But relevant. If it helps at all, I’m hoping to extend an invitation to you from the University of Canberra.. a work in progress, but jot it down in the CV as an invitation pending ;)