Who’s Tweeting Our Hashtag?

Last night, I put togther a quick video showing how to make a Yahoo Pipe that will find who’s been tweeting with a particular hashtag, in particular using the opened09 hashtag:

Here’s the pipe, deconstructed:

The idea behind this pipe in part came from a Dave Winer utility that will create an OPML feed linking to the Twitter feeds from the people that a named Twitter user is following (linked to from rssCloud news); my immediate reaction to that was: would an OPML feed linking to the feeds of everyone who had used a particular hashtag be useful? and this pipe is the result.

(Note to self: I really, really, really need to put a Pipes2OPML script together…)

Anyway, I thought it might make sense to generalise this pipe, and hopefully make it a little more useful, as well as showing a Pipes trick or two in some sort of appropriate context. So here are some problems with the above pipe that I’ll then show you how to fix…

– the pipe is hardwired to search for tweets containing #opened09; how can w generalise it to work with other hashtags/search strings?
– limited number of search results: the pipe only has access to the one hundred most recent opened09 hashtagged tweets; how can we get access to more?
– for a pipe that shows who has been using a particular hashtag, it might also be useful to see how many times they have used that hashtag?
– the pipe displays the hashtag twitterers in the order in which they first tweeted (a history based ordering); which is useful in one sense, but not in another. How else might we order the tweets? Recency of posting? Most active user of the hashtag?

So, first up, how do we generalise the pipe to work with other search terms? Simple, just add a user input for the search term and construct a URI using that term; then wire the URI in to the Fetch Feed block:

A form element appears on the pipes front page, and the search term is also passed via URIs that call the pipe:

Secondly, how do we increase the pool of hashtagged tweets? Simple: Twitter uses paged search results so we can pull in results from those other pages too…

So we can construct URIs for the second, third, fourth etc pages of search results too, and pull feeds in from all those pages:

The third item on our list was displaying how many times each user in the list had used the hashtag (based on the number of times they appeared in the search results sample, of course).

As @ostephens originally pointing out to me (at MashLib09), the Unique block actually counts the number of occurrences of the unique filter term:

We can display this in the title element using a very powerful construction… In the regular expression, rewrite the title string containing the value of the repeatcount variable. How? Like this – access the variable using the construction: ${VARIABLE_NAME}

So in the current example, we can get the value of the count using ${y:repeatcount}:

The final item on our improvements list was to try out some other orderings of the displayed tweets. At the moment, they are ordered according to the date on which each user first used the hashtag term (that is, within the sample of tweets we have pulled back from the Twitter search).

To change the order of names that are output from the pipe, we can use the Sort block. So for example, sort the users in terms of activity:

To sort the users in reverse chronological order (that is, according to who used the hashtag most recently), sort on the published attribute rather than on y:repeatcount.

You can now grab the RSS feed from the pipe and consume it in anything that accepts RSS, pull the JSON feed into your own web page if you’re a little more adventurous, or grab a Google homepage widget from the pipe’s homepage: Hashtag Twitterers Pipe

To tune the pipe to your own needs, you can also clone it from there and then modify your own version of it to your heart’s content:-)

Twitter Integration Means Delicious Social Bookmarking Site Gets, Err, Social…(?!)

In order to keep track of the dozens of snippets of potentially useful info I come across through m browser each week, I us the delicious online (social) bookmarking sit as a place to dump most of my bookmarks. Three or four times a week (maybe more?), I use my tags to rediscover things I remember bookmarking, and maybe once a month I actually use the delicious search option.

In order to save bookmarks, I used to use the integrated service provided by the flock browser, but in one particular update they made a change I really didn’t get on with (to the dialogue box, I think) and now I tend to use the delicious bookmarklet.

(I’ve also become pretty cavalier about which browser I use – I typically have Flock, Safari and Firefox open, and @dmje was hassling me all last week to use Chrome as well… – so with a bookmarklet on each browser, I get a consistent experience.)

As someone who used to send “FYI” emails out every so often, one of the ways I use twitter is to share potentially interesting or “of the moment” links; I also use a feedthru tag to post one or two links per day, (typically), to my blog sidebar (those links also gt integrated on a daily basis with my blog’s Feedburner feed). Note also that I rarely use the for: option on delicious, possibly because I don’t look at what’s been shared with me very often!

Anyway, one of the hassles with my workflow is the duplicated action required to both tweet and bookmark a link. But it seems that the delicious bookmarklet now has a sharing capability both within and without the delicious ecosystem. (I’m not sure if this is a Good Thing, or a delicious death throe?)

So for example, I can share a bookmark with my delicious network:

social delicious

Or tweet it (it’ll be interesting whether I adopt this workflow…):

So what message gets tweeted? The tweet message, of course:

Note that once the bookmark is saved, there is no evidence or history of it being tweeted. Nor are any of the tags used as hashtags. (If you add a hashtag to the tweet, I don’t think it gets added to the bookmark tags as a simple ‘dehashed’ tag (or hashtag.)

When the link is tweeted, a new delicious shortcode is used:

One Bad Thing about the Twitter integration – you have to provide your twitter credentials. So what the f**k is wrong with OAuth?!

As well as the new social/network amplification options in the bookmark dialogue, there’s also been a revamp (I think) of the search facility:

As well as suggested terms, an improved search display over your own bookmarks, your network’s, or everyone’s, there’s an ability to filter the results by tag. I have to admit I expected live AJAXy UI updates – I didn’t see the effect of filtering by tag unless I clicked the Search button again – but it’s maybe still early days and the live reflow may yet appear. (Or maybe it is there already and just broken for me at the mo?!)

I’m not sure how useful the volume display will be (memories of Google trends etc, there), especially as it only works when you only have one group selected (i.e. only one of my bookmarks, or my network’s, or all of them) and doesn’t reflow when you change the selection? I also wonder how well the ads will fare against the user generated links?

Anyway this is starting to look like it could become quite a powerful search tool, so maybe I need to start growing my delicious network and evangelising once more… (Just a quick note to self – if I do a social bookmarking workshop again, I need to update my slides [uploaded 3 years ago? Sheesh…] ;-)

And as for the twitter integration – I think I’ll give it a go…

PS on the search engine front, I was thinking over the weekend how the mythical ‘social search engine’ that people were trying to hype a year or two ago has actually appeared. But rather than arising out of ‘dead links’ posted to delicious, it’s a live ‘person inside’ application: Twitter.

PPS at last there’s an official announcement post: New and Delicious: Search, Tweet, and Discover the Freshest Bookmarks

Brand Association and Your Twitter Followers

One of the thing’s that Martin appears to have been thinking a lot about lately are metrics for rating ‘digital scholars’, i.e. those of us who don’t do any of the reputation bearing thing that traditional academics do, (though whether that’s because we’re not very good at those things is not for me to say;-)

So for example, in The Keynote Equivalent? he reviews the notions of reputation and impact factor, and in Connectors versus outputs he calls on some really dubious social media buzz metrics to raise the more far more valid issue of how we measure influence within, and value the contributions made to, a social network (peer community?) in order to recognise the extent of someone’s influence within that network from outside of it.

Using Twitter as a base case, one of the many interesting features of Twitter’s ‘open privacy’ model is that in most cases it’s possible for you to look at someone else’s followers to see who they are.

The value of that network to an individual is at least twofold – firstly, as a source of information, observations, news and feedback to you as the person at the centre of your own network; secondly as an amplifier of your own ego broadcast messages. (There are other benefits of course – like being able to see who is talking to whom about what.) You may also feel there is some benefit to just having a large number of followers, if only in the bragging stakes.

That is, the more followers the better, right? It’s bound to be good for my reputation, if nothing else, surely…?

Well….. maybe not…?

Consider these two questions:

who follows you? if I look at your followers what can I tell about you, from them?
what is your blocking policy? who you block is just as much a part of the way you manage your network as the people you actively follow.

As far as my own Twitter network goes, I am on a follow:followed ratio of about 1:4. That is, approximately four times as many people follow me as I follow back. For every 10 or so new followers I get, I block one or two.

I check my followers list maybe once every two or three days, which lets me keep up with the pruning on just one or two screens of followers using the Twitter web interface. If the name or avatar is suspect, I’ll check out the tweets to see if I want to block. (I really miss the ability to hover over a person’s name and get a tooltip containing their bio:-( If the name or avatar is familiar or intriguing, I’ll check the tweets to see if I’m going to follow back (maybe 1 in 20? Following back is not the main source for me of new people to follow – you’ll have to get to me another way;-).

The people I block? People who’s tweets are never replies, but who just tweet out advertising links all the time; Britney, whatever she happens to be sucking or loving at the time; product tweeters; and so on. If you’re following lots of people and only followed by a few? Not good – why should I follow you if no-one else does? If you’re following lots of people and are followed by lots of people? Also not good: either you’re a spammer being spammed back, or you’re an indiscriminate symmetric follower so why should I trust you, or you’ve so many followers I’m not going to get a look in. If I’m not sure about a new follower, it’s 50/50 that I’ll either block them or not, so there may well be the odd false positive amongst the people I’ve blocked (if so, sorry…) And why do I block them? Because they add no value to me… Like junk mail… And because by association, if you look at my followers and see they’re all Britney, you’ll know my amplification network is worthless. And by association… ;-)

The people I follow? People I’ve chatted to, have been introduced to through RTs, or via interesting/valuable multiaddressed tweets that include me; people who appear not to be part of any other network I follow (or who might add value in a sphere of influence or interest that I don’t feel I currently benefit from), and so on.

And the people I don’t follow but don’t block (i.e. the majority) – nothing personal, but I only have so many hours in the day, and can’t cope with too many new messages every update cycle in by twitter client!

So all this might sound a little bit arrogant, but it’s my space and it’s me that has to navigate it!

PS just by the by, it struck me during an exchange last week that networks can also act as PR channels. A tweet went out from @ruskin147 asking if anyone knew anyone “who can analyse how viral emails,campaigns etc, can knock a firm off course?” Now I should probably have recommended someone from the OU Business School, because I think there is someone there who knows this stuff; but they’re not part of any of my networks so I’d have to go and search for them and essentially recommend them cold. So instead I suggested @mediaczar (who blogs under the same ID) because he’s been sharing code and insight about his analysis of connectivity and the flow of ideas across social networks for the PR firm (I think?) he works for. (Some irony there, methinks?;-) And it turned out that the two of them hooked up and had a chat…

So why’s that good for me? Because it strengthened the network that I inhabit. It increased the likelihood of those people having an interesting conversation that I was likely to also be interested in. I get value not just from people telling me things, but also from people in my network telling each other things that I am likely to find interesting.

And as a spin-off, it maybe increases my reputation with those two people for having helped create that conversation between them?

In terms of externally recognised value though? How are you going to measure that, Martin?

See also: Time to Get Scared, People?

Hyperlocal Twitter Trends

Just by the by, I idly tweeted last week along the lines of “does anyone know of a twitter trends service that identifies trending topics within a particular region or locale?”.

I didn’t receive any links to such a service at the time (and didn’t build the service myself…) but it strikes me that this could be a really useful hyperlocal news service?

An alternative might be to find ‘trending locations’ or ‘trending places’ rather than trending hashtags or topics, so if there is a sudden flurry of tweets in a particular area, it could get flagged (does Twitter this anyway with its trending topics?).

The locus of the trending regions need not be limited to ‘a circle within fifty miles of a point’ either; they could (with lots of computing power;-) be points along a line, such as a road, for example.

As an asymmetric follower type (I have many more followers than people I follow on Twitter), it might also be handy to be able to see trending topics across the people who follow me, just in case the sampled population that I do follow aren’t a fair sample of the people who follow me…

Just a thought… now back to the jet lag :-(

[See also: Mapping Realtime Events on Twitter and this Simple Embeddable Twitter Map Mashup]

UPDATE: with the release of a Twitter geo api, it didn’t take long: Trendsmap

Mapping Realtime Events on Twitter

One of the nice things about blogging in WordPress is the dashboard report that shows which other blog pots are linking in to your blog. One of my posts that’s had a couple of incoming links lately is Simple Embeddable Twitter Map Mashup, firstly from this post on TweetMapping Archaeology and then from Twitter Watermain Mapping – Part Two.

This latter post – plotting out tweets around the occurrence of breaks in watermains – also plots out a map showing people twittering about stormwater.

Which got me thinking, how about a Twittermap to plot tweets about electricity, gas or water being cut off?

By altering the search term, you can search for other events, such as earthquake or bee swarm:

If you want to search around a particular location, then this pipe may be more useful- locale based twittermap (the default search is for the word bid, but it works equally well if you’re wondering where the fire is):

Finally if you’d rather just use the URL for a Twitter search feed as the basis for a map, this pipe should do: map a Twitter search feed URL.

Anti-tags and Quick and Easy Block (Un)commenting

Looking back over the comments to @benosteen‘s post on Tracking conferences (at Dev8D) with python, twitter and tags just now, I noticed this comment from him replying to a comment from @miaridge about “app noise” appearing in the hashtag feed:

@mia one thing I was considering was an anti-tag – e.g. #!dev8d – so that searches for ‘dev8d’ would hit it, but ‘#dev8d’ shouldn’t.

The other tweak to mention is that booleans work on the twitter search:

‘#dev8d -from:randomdev8d’ would get all the #dev8d posts, but exclude those from randomdev8d.

Likewise, to get all the replies to a person, you can search for ‘to:username’, handy to track people responding to a person.

Brilliant:-)

Note also that one thing worth bearing in mind when searching on Twitter is that a search for @psychemedia is NOT the same as a search for to:psychemedia. That is, those two searches may well turn up different results.

The “to:” only searches for tweets that START with “@pscyhemedia”; so id @psychemedia appears elsewhere in the tweet (e.g. “waht the ??? is @psychemedia talking about?”), the “to:” search will not find it, whereas the “@” search will.

Why’s this important? Well, a lot of people new to using Twitter use the Twitter website interface to manage their interactions, the the “Replies” screen is powered like the “to:” search. Which means if someone “replies” to you in a “multiple addressee” tweet – e.g. “@mweller @psychemedia are you gonna make some more edupunk movies?”, then if you’re not the first named person, the @Replies listing won’t show the tweet… the only way you can discover them is to search twitter for “@psychemedia”, for example.

The Twitter advanced search option to search for posts “Referencing a person” is simply a search of the @person form.

(Note that Twitter search lets you subscribe to search results – so you can always subscribe to an ego search feed and receive updates that way; or you can use a client such as Tweetdeck which runs the search automatically.)

(I’m not sure what happens if someone actually replies to one of your tweets and then prepends some text before your name? Will twitter still spot this as a reply? If anyone knows, can you please comment back?)

Just by the by, the “anti-tag” trick reminds me of this code commenting pattern/trick (I don’t remember where I came across it offhand?) that makes it easy to comment and uncomment blocks of code (e.g. in PHP or Javascript):

Before the comment…]
/*
This will be commented out…
//*/
After the comment…

To uncomment out the block of code, just add a leading “/” to the first comment marker to turn it into a single line comment:

Before the comment…]
//*
This will NO LONGER be commented out…
//*/
After the comment…

The block comment closing tag is now itself single line commented out.

(I seem to remember it was described far more eloquently than that when I came across it!;-)

PS Ah ha, here we are – where I first saw the commenting trick: Every keystroke is a prisoner – a neat commenting trick.

Twitter Powered Youtube Subtitles, Reprise: Anytime Commenting

One of the things that attracts me to serialised feeds (as well as confusing the hell out of me) is the possibility of letting people subscribe to, and add, comments in “relative time”…

… that is, as well as viewing the content via a serialised feed, the comments feed should also be serialised (with timestamps for each comment calculated relative to the time at which the person commenting started receiving the serialised feed).

Applying this to the idea of tweeted Youtube movie subtitles (Twitter Powered Subtitles for Conference Audio/Videos on Youtube) in which every tweet made during a presentation at or around that presentation becomes a subtitle on a recording of that presentation, it strikes me that a similar model is possible.

That is, different individuals could watch a Youtube video at different times, tweeting along as they do so, and then these tweets could be aggregated according to relative timestamps to provide a single, combined set of subtitles.

So how might this work in practice? Here’s a thought experiment run through…

Firstly, it’d probably be convenient to set up a twitter account to send the tweets to (say @example, for example).

Create a tag for the video – this could be something like #yt:tBmFzF8szpo for the video at http://www.youtube.com/watch?v=tBmFzF8szpo.

(Alan Levine reminded me about flickr machine tags earlier today, which are maybe also worth considering in this respect, e.g. as a source of inspiration for a tagging convention?)

Grab a ctrl-C copy of the phrase @example #yt:tBmFzF8szpo for quick pasting into a new tweet, and then start watching the video, tweeting along as you do so…

To generate your subtitle feed, you can then do a search based on Tweets from your username (which would be @psychemedia in my case) to e.g. @example, with hashtag #yt:tBmFzF8szpo, and maybe also using a date range.

(You could augment the Yahoo pipe I used in the twitter subtitle generator proof of concept to remove the hashtag when generating the feed used for subtitling?)

The actual subtitle file generator could then pull in several different subtitle feeds from separate people, relativise their timestamps relative to the time of the first tweet (which could maybe use a keyword, too – such as “START”: @example START #yt:tBmFzF8szpo;-) and then produce an aggregated subtitle feed.

As more people watched the video (maybe including the subtitles to date), their feeds could be added to the aggregating subtitle file generator, and the subtitle file updated/refreshed.

Individuals could even rewatch the video and create new feeds for themselves to join in the emerging conversation…

(Okay, so it’s maybe slower than just reading through the comments, having to replay the video in real time to read the tweets, but this is a sort of thought experiment, right, albeit one that can be implemented quite easily…;-)

PS In one of the comments to Show and Translate YouTube Captions Matt Cutts gave an example of a URL that “will search for the word “china” in videos with closed captions” [ http://www.youtube.com/results?closed_captions=1&search_query=china ] (although I’m not sure how well it works?).

So I’m thinking – if live tweets from an event can be associated with a video of an event (maybe because the video is posted with a link to a (now out of date!) upcoming record for that event in order to anchor it in time) then being able to search the tweets as captions/subtitles provides a crib for deeplink searching into the video? (But then, I guess the Goog is looking at audio indexing anyway?)

PPS I just came across another tool for adding subtitles to Youtube videos, as well as videos from other online video sites – overstream.net:

It’s worth looking at, maybe?

PPPS see also Omnisio, a recent Google acquisition that offers “select clips from videos you find on YouTube and other video sites, and easily post them on your profile page or blog. Even better, you and your friends can add comments directly in the video!”.

And there’s more: “With Omnisio you make and share your own shows by assembling clips from different videos.” Roll on the remixes :-)

PPPPS Martin implemented anytime commenting