OUseful.Info, the blog…

Wouldn’t it be handy if you could use Yahoo Pipes as code free rapid prototyping development environment, then export the code and run it on your own server, or elsewhere in the cloud? Well now you can, using Greg Gaughan’s pipe2py and the Google App Engine “Pipes Engine” app.

As many readers of OUseful.info will know, I’m an advocate of using the visual editor/drag and drop feed-oriented programming application that is Yahoo Pipes. Some time ago, I asked the question What Happens if Yahoo Pipes Dies?, partly in response to concerns raised at many of the Pipes workshops I’ve delivered about the sustainability, as well as the reliability, of the Yahoo Pipes platform.

A major issue was that the “programmes” developed in the Pipes environment could only run in that environment. As I learned from pipes guru @hapdaniel, however, it is possible to export a JSON representation of a pipe and so at least grab some sort of copy of a pipes programme. This led to me doodling some ideas around the idea of a Yahoo Pipes Documentation Project, which would let you essentially export a functional specification of a pipe (I think the code appears to have rotted or otherwise broken on this?:-(

This in turn led naturally to Starting to Think About a Yahoo Pipes Code Generator, whereby we could take a description of a pipe and generate a code equivalent version from it.

Greg Gaughan took up the challenge with Pipe2Py (described here) to produce a pipes compiler capable of generating and running Python equivalents of a Yahoo pipe (not all Pipes blocks are implemented yet, but it works well for simple pipes).

And now Greg has gone a step further, by hosting pipe2py on Google App engine so you can make a working Python backup of a pipe in that environment, and run it: Running Yahoo! Pipes on Google App Engine.

As with pipe2py, it won’t work for every Yahoo pipe (yet!), but you should be okay with simpler pipes. (Support for more blocks is being added all the time, and implementations of currently supported blocks also get an upgrade if, as and when any issues are found with them. If you have a problem, or suggestion for a missing block, add a comment on Greg’s blog;-)

(Looking back over my old related posts, deploying to Google Apps also seems to be supported by Zoho Creator.)

Quite by chance, @paulgeraghty tweeted a link to an old post by @progrium on the topic of “POSS == Public Open Source Services: … or User Powered Self-sustaining Cloud-based Services of Open Source Software”:

How many useful bits of cool plumbing are made and abandoned on the web because people realize there’s no true business case for it? And by business case, I mean make sense to be able to turn a profit or at least enough to pay the people involved. Even as a lifestyle business, it still has to pay for at least one person … which is a lot! But forget abandoned … how much cool tech isn’t even attempted because there is an assumption that in order for it to survive and be worth the effort, there has to be a business? Somebody has to pay for hosting! Alternatively, what if people built cool stuff because it’s just cool? Or useful (but not useful enough to get people to pay — see Twitter)?

Well this is common in open source. A community driven by passion and wanting to build cool/useful stuff. A lot of great things have come from open source. But open source is just that … source. It’s not run. You have to run it. How do you get the equivalent of open source for services? This is a question I’ve been trying to figure out for years. But it’s all coming together now …

Enter POSS

POSS is an extension of open source. You start with some software that provides a service (we’ll just say web service … so it can be a web app or a web API, whatever — it runs “in the cloud”). The code is open source. Anybody can fix bugs or extend it. But there is also a single canonical instance of this source, running as a service in the cloud. Hence the final S … but it’s a public service. Made for public benefit. That’s it. Not profit. Just “to be useful.” Like most open source.

Hmmm….. ;-)

One of the challenges I’ve set myself this year is to write some sort of book about Yahoo Pipes. Reading Presentation Zen three or four weeks ago, I started to imagine the form such a book might take. What I aspired to was something uncluttered, something that would contrast with the typical confusion of words and ideas that tend to end up being dumped into OUseful.info; something like an artistic recipe book, perhaps, or an art gallery catalogue; the form should be decomposable, allowing sections to be removed or updated without too many side effects on the rest of the work; and the authoring environment should complement the the publication environment, enforcing constraints of the medium the book would be published into.

In short, something like Powerpoint done well, but for print rather than screen.

It seems (of course!) that Tim O’Reilly had already executed a similar idea in the form of the Twitter Book, as John Naughton pointed out to me a couple of days later.

You can read more about O’Reilly’s take on the philosophy behind this sort of representation in Reinventing the Book in the Age of the Web.

Anyway, I spent a weekend doodling ideas, and then left it a couple of weeks. Now I’m looking at it again, and I’d appreciate your comments on whether this sort of presentation works for you, (and if not, why not?), how it might be improved, how it might be simplified (but remain accessible to a novice) and so on. The numbering scheme used is not related to pages – instead, each “point” I make has a number, and these are referred to from the index (I drew inspiration for this sort of numbering from The Pengin Cookery Book). Comments on the level at which the technical content is presented, and the way in which I have started trying to develop a narrative, will also be appreciated.

I originally thought that the “book” should be printed in an A4 landscape form, but then I started to wonder whether two landscape A4 pages could be combined into a portrait A4 page. The font size is problematic, and the I don’t think the same layout works for the landscape vs. portrait view, at least, not as it currently stands.

Anyway, here are the landscape and portrait versions. I don’t think they work as embedded content, which is a shame, but they weren’t written for that sort of medium, so it’s to be expected.

(If you are reading this in a feed reader, you will probably need to click through to the original post in order to see the embedded documents.)

Please bear in mind, too, that I’m not a designer (this much will be be obvious), but that I do think design could play a large part in making this approach usable.

Please feel free to add your comments below:-)