So Google Loses Out When It Comes to Realtime Global Events?

There’s been quite a few posts around lately commenting on how Google is missing out on real time web traffic (e.g. Sorry Google, You Missed the Real-Time Web! and Why Google Must Worry About Twitter). Stats just out showing search’n’twitter activity during Obama’s inauguration yesterday show how…

First up, Google’s traffic slump:

And then Twitter’s traffic peak:

And how did I find out? Via this:

which led to the posts containing the traffic graphs shown above.

And I got a link to that tweet from Adam Gurri who was responding to a tweet I’d made about a the Google traffic post… (which in turn was a follow-up to a tweet I posted about whether anyone “turned on a TV to watch the US presidential inauguration yesterday?”)

Adam also pointed me to this nice observation:

And here’s how a few people responded to how they watched the event (I watched on the web):

So, web for video, broadcast radio for audio and Twitter for text. And TV for, err… time for change, maybe?

Another Nail in the Coffin of “Google Ground Truth”?

So we all know that the Google web search engine famously (and not just apocryphally) returns different results from it’s different national representations (google.com. google.co.uk, google.cn, etc.)…

…and hopefully we all know that if you are signed in to Google when you run a search, the default settings are such that Google will record your search and search results click-thru behaviour using Google Web History, and then in turn potentially use this intelligence to tweak your personal search results…

…and depending on how much you’ve been paying attention, you may know that Google Search Wiki lets you “customize search by re-ranking, deleting, adding, and commenting on search results. With just a single click you can move the results you like to the top or add a new site. You can also write notes attached to a particular site and remove results that you don’t feel belong. These modifications will be shown to you every time you do the same search in the future.

Well now it seems that Google is experimenting with Google Preferred Sites, which let selected guinea pigs “set your Google Web Search preferences so that your search results match your unique tastes and needs. Fill in the sites you rely on the most, and results from your preferred sites will show up more often when they’re relevant to your search query” (see the official support page here”: Preferences: Preferred sites).

So the next time you give someone directions to a website using an instruction of the form “just google whatever, and it’ll be the first or second result”, bear in mind that it might not be…

(For what it’s worth, I run a cookie free, never logged in to Google browser to compare the results I get from my logged in’n’personalised Google results page and a raw organic” Google results page.)

Discovered Custom Search Engines

Although Google manages to serve up pretty good results most of the time, sometimes it makes sense to give the search engine a hand by limiting the search to only provide results from a particular set of pages, or domains. So in this post I’ll describe a couple of “emergent” or “discovered” custom search engines that are available in tools you might already use.

(Custom search engines provide one way of achieving this, of course – set the limits over which you want results returned from, et voila… But creating custom search engines, as such, is not necessarily something that would occur to most people.)

Let’s start with delicious, the social bookmarking service, in which users bookmark links to delicious, with one or more tags.

Did you know that there are now a range of tools within delicious that let you search over the titles and descriptions of different sets of bookmarks?

If you pick a particular user, the default Search these bookmarks search will just search over the title and description fields of the bookmarks saved by that user. If you further limit the view of the bookmarks to those tagged in a particular way by a particular user, then the Search these bookmarks search will be limited to just those bookmarks. In other words, Search these bookmarks is context sensitive to the user, tag or user’n’tag combination that is currently selected.

(Remember that the full text of the bookmarked pages is not being searched – only the bookmark title and description fields – which is one good reason why it makes sense to fill in a bit of description about every bookmark you make: it makes (re)discovery of links at a future time easier…)

So where else do people create there own resource collections, or resource feeds Google Reader, maybe?

And as it happens, another emergent, “auto-created” custom search engine can be found just there:

The Google Reader search provides a blogsearch facility that lets you limit your search to the content of the RSS feeds you subscribe to in a variety of ways: the content of all your feeds, the content of the items you’ve read, the content of feeds bundled in various folders, and so on.

So for example, you could bundle a set of RSS feeds together in a single folder, and then, as if by magic, you have a custom search engine that searches over just the contents of those feeds.

With Google’s “official” blogsearch tool no longer functioning as such (rather than just indexing feed content – that is, just actual blog posts – it appears to be indexing blog web pages, so you get contaminated results that may only be a “hit” because your query was matched by sidebar content or other blog website fluff), the Google Reader search tool goes back to basics…

…the only problem is, that so far as I can tell, there is no way to subscribe to the results of any of these searches, and there is no published (or community documented) API for the Google Reader search facility… (so if someone can watch the AJAX calls and produce one, I’d be really grateful :-)

(By the by, can you define filters on folders in Google Reader, a bit like iTunes Smart Playlists?)

See also: Search Hubs and Custom Search at ILI2007.

PS if you are looking for an effective blogsearchengine, Icerocket has been grabbing the buzz lately…

A Couple of Twitter Search Tricks…

Just a quickie post, this one, to describe a couple of Twitter search tricks’n’tips (which is to say, this is an infoskills post, right?;-)

You can find the Twitter search tool at http://search.twitter.com. I actually call it in my browser using the keyword “tw” associated with a Firefox Keyword Search.

Link search: if you’re in the habit of searching social bookmarking sites such as delicious for useful links, whether by pivoting around particular tags or tag combinations, or by using the delicious search box, you might also be interested in searching for tweeted links. Here are a couple of ways of doing it…

The “official way”, using a Twitter advanced search form – just select the “contains Links” option.

This invokes a special search limit, filter:links, which you can also enter directly into the Twitter search box:

If for any reason that search limit isn’t working, here’s a workaround that makes use of Twitter search’s partial string matching capability:

Fan out: see which of your tweets have been retweeted by others (maybe;-)
This trick relies on a convention that has emerged in which Twitterers use the pattern along the lines ofRT @username “the original tweet”.

(See also the ReTweetist service, which will plot which of your messages have been retweeted, as well as the most popular current retweets.)

Also remember that you can subscribe to an RSS feeds of saved searches based on these query types:

Locale Based Searches
Want to know who’s recently been twittering (possibly) from nearby a particular location? Set the location options in the advanced search form, and run an otherwise empty query (i.e. no search terms in the search box).

So for example:

Now it used to be that you could search people’s biography or location strapline in Twitter, and find people to follow that way (that’s how I found several fellow Isle of Wight twitterers) , but that doesn’t seem possible using the “Find People” service at the moment? (And I can’t check to make sure, because the “Find People” service is temporarily stressed (i.e. down) again…).

So here’s a Google hack way round finding Twitterers from a particular location – construct a query of the form:

http://www.google.com/search?q=location+wight+site%3Atwitter.com+-inurl%3Astatus+-intitle%3Awight

This works as follows – look for the search term, on twitter.com (site:twitter.com), but try not to return results from tweets (-inurl:status) or where part of the location appears in the user’s Twitter ID (-intitle:wight). If an individual’s page is indexed when there’s a tweet showing that contains the search term, then you may get the page returned as a result. But more likely you’ll only get results from pages where the search term is always present, such as when it’s part of a person’s bio… In a sense, this is a bit like indexing a fixed set of web search engine indexable, on-page, bio/location meta-data.

[UPDATE: looking at the results preview, if we search for “Location Isle of Wight” we can probably filter the results even further:
“location isle of wight” site:twitter.com -inurl:status -intitle:wight

And as @daveyp suggests, we can also search for institutional allegiance within a profile, eg site:twitter.com -inurl:status -intitle:huddersfield location huddersfield university]

(You can do something similar to stalk people on MySpace.)

For more Twitter search tricks , check out the Twitter advanced search form, or have a creative play in Google;-)

OU Participation in the Ofcom PSB Review

So the Ofcom PSB review is due out in the next few days, and Carter’s Digital Britain report a few days after that…

Just because, I had a little dig around to see what representation the OU might have made, not least because of our involvement with traditional broadcast via the OU relationship with the BBC… (Note to self: check whether these BBC Commissioning – Open University – Rights Guidelines are current?)

Here’s a link to the OU’s response to Phase One of the Ofcom PSB review from the middle of last year: OU Response to Ofcom PSB Review Phase 1 (PDF) (read it on Scribd (maybe?)). (I couldn’t find a response for pahse 2?)

If you want to know what it said… well, you’ll just have to read the response yourself (it’s not too long). One thing I did find particularly interesting, though, was that there was no response to the question “8i) What do you think is the appropriate public service role for Channel 4 in the short, medium and long term? What do you think of Channel 4’s proposed vision?”.

Given that the OU was part of a consortium that (unsuccessfuly) bid to take over the running of Teachers TV last year, I’d have thought we might have an interest in who was involved in PSB in a wider sense (and what relationship the OU might have with them?)

And given one of the apparently mooted options for the future of Channel 4 is some sort of Channel 4 partnership with BBC Worldwide, what if part of that option suggested that the OU pays Channel 4, rather than the BBC, to produce and broadcast OU programmes?!

And as for contributions to the Carter report? I couldn’t find any public responses – though with one of the anticipated sections of Digital Britain covering the questions of intellectual property rights and their enforcement on the internet, there could be a potential “new revenue stream” for the OU exploiting our rights clearance experience, particularly as other universities seek to publish their teaching materials on the web?

PS As a quick refresher, here’s a quote from the OU charter about broadcast: “The objects of the University shall be the advancement and dissemination of learning and knowledge by teaching and research by a diversity of means such as broadcasting and technological devices appropriate to higher education, …“.

How Not to Launch a Website, Reprise… (incl. the OUseful.info Blogging Policy)

So it seems that there’s a backstory to the launch of Re-launch/Outsmart the Recession (as, err, critiqued in “Re-launch” – How Not to Launch A Website): the planned release date for the site was brought forward to fit in with a PR opportunity (the invitation of our VC to Gordon Brown’s jobs summit), which resulted in a Friday afternoon request to get the site live before the following Monday morning (even though at the time the site was nowhere near the state it would normally be expected to be in for a launch)…

…and, as is the way of these things, some of the planned tidying-up didn’t make it through in time to the site that went live.

And then I blogged about it…;-) So apologies to the team behind the site for any distress caused – maybe there’s something from that confusion that we can beneift from as a learning institution?!;-)

Anyway, given all that, I guess now is as good a time as any to complement the How OUseful.Info Operates… post from last year with a few remarks about the, err, “editorial policy” I sort of apply to OUseful.info, the blog…?

Firstly, I blog about things that interest me – OUseful.info is my notebook, a record of things I’ve tried to do, seminars/talks I’ve been to, and so on.

Secondly, OUseful.info is probably the primary way in which I engage with my “peers”, in whatever ad hoc wider academic/ed tech community it is that I am a member of.

Thirdly, OUseful.info is an inside-outside look at the OU in particular, and bits of HE and the academic library sector in general. This is probably the most contentious aspect of the blog, and the one that causes internal readers to twitch a bit, so it’s probably worth clarifying the stance I have towards writing OU focused posts:

– no ad hominem attacks, there’s just no point;
– stuff that appears on public OU websites is generally fair game; some of the time I will try to get in touch with people who developed a site, or a page, via email to point out my “concerns”, rather than writing a blog post. In those cases, if a site remains broken for a week or two, (and is in public), then it’s “in play”;
– stuff that appears on internal, behind the firewall sites is more of a moot issue. Problems that I know are widespread, either within the OU, or that are likely to be familiar to people from other institutions, are in play. Like the OU intranet search. Which sucks…. big time. (I dread to think how much time is wasted by people not finding things they are looking for on it.)
– OU processes are fair game, except maybe where the process is best not talked about in public for whatever reason (legal, ethical, personal identifiability or commercial reasons, for example). Many OU processes formalise a snapshot of how the OU did things at some time in the past, when things were different. For example, the legacy of T171 can be seen in the design of many of our current online courses, such as in the way we present the materials to students, and in the design of the OU-XML schema used in the structured authoring workflow. Much OU work is only possible because people know how to get things done outside (or in spite of) the official processes, which I suspect is one reason why many appointments to OU positions go to internal applicants.
– “roles” and decisions are sometimes in play, for example where the role of a person in a particular workflow is constrained by the workflow, internal management structures and so on, to make a decision in a particular way (or default to a particular decision when it’s not clear what the actual answer should be). For internal readers of the blog, sometimes posts that focus on these issues are seen as ad hominem attacks – they’re not intended to be; they’re intended to sympathise with the person who made a decision a particular way, particularly ones they may not personally agree with, or ones they feel they were not really in a position to answer or explore in a properly informed way, whether through lack of knowledge, training, an appreciation of the “wider context”, and so on. If the “system” forces a person to answer a particular sort of request in a particular, default reasoning way (particularly when then the original premises or assumptions behind those default reasoning models are no longer valid), then I feel almost duty bound to criticise it! Sometimes, I try to raise “issues” internally, via email, meetings and so on. Sometimes I resort to the blog, because then I know someone will be forced to respond…
– sometimes a post hits a nerve with internal readers, and they don’t know how to respond, or don’t feel they are in a position to respond, for example because of how a line manager might respond to their response (or a feeling that it’s up to the line manager to handle it). In terms of how to respond, there are a couple of good ways – via a public blog comment or tweet (I’m @psychemedia), or via a private email. I’m happy to post corrections and clarifications, though I’ll rarely pull a post (one it’s posted, copies are out there anyway). I’m happy to use different devices to reference internal sources (or not, as required;-) in posting clarifications, including posting updates to posts, comments to posts, tweets about posts and maybe even follow-on posts;
– sometimes internal readers feel they can’t tell me something “because you’ll blog it”. Not true. I don’t blog lots of things (such as about the Register debacle last year, for example… doh!). I draft posts for some things then delete them (maybe because I can’t find a way of posting something that would almost definitely be seen as an ad hominem attack). And I respect embargoes (for example, holding off on posts about Platform, or other OU sites I’ve seen in staging). Recalling a point made earlier, though, if a site is in public, it’s potentially in play…
– sometimes I use content from emails, or email conversations, in posts. But I try to do this in a fair and reasonable way, and employ various literary devices and appropriate citations (or not!) for including such content when the words aren’t mine, or when something I’ve said is in response to, informed by, or building on something someone else said. If I’m uncertain as to whether I can blog something covered in a meeting, seminar, conversation or email, I will typically ask them if it’s “okay to blog that?”. (You can trust me…. heh heh;-)
– very occasionally I may go a little too far, and post something I’m maybe not totally comfortable with. (In such cases, I may also take a sounding from other OU bloggers and staff before publishing a post.) These rare posts are done deliberately, in order to test the edges of what’s acceptable, and they can make me feel as queasy as the people reading them! That’s how you know the post is on the edge, right?!
– and finally: whilst OUseful.info is a personal blog, and not an official OU communications channel, I am happy to get the message out about OU initiatives if they are likely to be of interest to me now (or in the future) and/or the readers I think I have. And as mentioned above, I’m happy to respect embargoes…. but just remember, the posts contained in this blog express the personal opinion of the author, and the author’s decision is final about what actually goes into a post..;-)

Change the Law to Fit the Business Model

I’ve just been watching the following video from the Open Rights Group (ORG) on copyright extension, and realised something…

[Via Ray]

…something that’s probably obvious to anyone who lobbies against this sort of thing (extending copyright of works to prevent them from entering the public domain), but came like a doodah out of the whatsit to me…

The companies that are lobbying for copyright extension built their business models around the idea that “our artists’ work is in copyright, so we can exploit it like this and this and that.”

But as these companies are now getting on a bit, that’s not true any more. They need a business model built around the idea that “we are purveyors of in and out-of-copyright material”.

[As prompted by a clarification request from @mweller: the industry’s business model is broken in other ways, of course, not least the changing costs of reproduction and distribution. My “insight” was limited to a realisation (that works for me;-) that lobbying for copyright extension is the industry’s attempt to protect a revenue stream built on an incorrect assumption – i.e. the existence of a never-ending revenue stream from an ever-growing in-copyright back catalogue, or maybe the assumption that growth in new release sales would make up for loss of copyright based revenues from older stock? That’s probably not right though, is it? It’s probably more a blend of complacency – living off the fat of Beatles and Cliff Richard early-recording revenues, not being able to develop the desired level of new artist revenues, and the dreadful realisation that large amounts of moneymaking product is about to go out-of-copyright, whereas you might once have expected the longterm sales value of that product to dwindle over time? Like it did with Shakespeare… Err…?]

[Cf. also software companies, where the value generating life of a piece of software only extends as far as the current, and maybe previous, version, rather than version 1.0a of a product that’s now at version 10? Though thinking through something Alma H said to me yesterday, in a slightly different content, I guess if the media companies followed the lead of the software industry, they’d just delete/remix/re-release the same old songs and keep refreshing the copyright with every new “release”!]

But that’s too hard to imagine – so it’s easier to lobby to changes in the law and keep the same business model ticking over.

Cf. also academia and library sectors, which were built around the idea that access to high quality information and knowledge was scarce. Oops…