Using YQL With Yahoo Pipes

A couple of days ago, @mikechelen asked:

“can yql plug in to pipes for improved development, compared with other cloud platforms that accept standard languages?”

where YQL is the Yahoo Query Language, a SQL like query language that can run queries on data pulled in from all over the web…

There are a couple of ways at least of doing this: a) calling YQL from Yahoo Pipes; b) calling Yahoo Pipes from within a YQL query.

First up, calling YQL from Yahoo Pipes, using the pipes YQL block and a trick I learned from @hapdaniel that lets me run a query on a couple of Google spreadsheets, where the results from one of the spreadsheets are subselected based on results of a query to a second Google spreadsheet:

select * from csv(2,500) where url =’http://spreadsheets.google.com/pub?key=phNtm3LmDZEM6HUHUnVkPaA&gid=47&output=csv’ and col4 > ’70’ and col1 in (select col1 from csv(2,500) where url = ‘http://spreadsheets.google.com/pub?key=phNtm3LmDZEM6HUHUnVkPaA&gid=40&output=csv’ and col4 > ’70’)

The second approach is to run a YQL query, e.g. with the YQL console, that calls on the JSON output of a Yahoo pipe (in this case, I just happen to be displaying the results from the pipe shown above. That is, a pipe that itself embeds a different YQL query).

http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20json%20where%20url%3D%22http%3A%2F%2Fpipes.yahoo.com%2Fpipes%2Fpipe.run%3F_id%3D0IpRD0sP3hGFBtfI3XBDOQ%26_render%3Djson%26minArchitecture%3D70%26minPlanning%3D70%22&format=xml

So, there you have it – how to embed YQL in a Yahoo pipe, and how to call a Yahoo pipe from YQL…

PS See also HTML Tables and the Data Web and the Google Visualization API
Query Language, which apparently lets you “perform various data manipulations with the query to the data source. The Query Language does not depend on the implementation of any specific data source. These data manipulations are performed by the data source server, reducing the need to perform data manipulations and formatting by developers on the client.”