How to use Yahoo Pipes to aggregate Feeds (Flickr and Twitpic)

RSS feeds are everywhere nowadays.  Nearly every other website spits out these lists of information and the large majority of people don’t use them.  Only a minority wake up every morning and check their mass of feeds to learn what’s going on but what about everyone else? Yahoo Pipes has been out since 7th February 2007 but in those two years it seems to have existed quietly. Well even though all the aggregation, mixing and sorting of web data sounds complicated, with Yahoo Pipes it’s simple.

Personally I use iGoogle to link some of my favourite RSS feeds together to read as my homepage but recently I was met with a dilemma.  The growing emergence of Twitpic means my usual photo site of Flickr is suffering.  Now I’m taking more photos on the go and uploading them to Twitpic through Twitterific on my iPhone because it’s just so much easier. However this means for my audience they have to follow my Twitter 24/7 to see the pictures as currently there is no easy way to sort them out and many people won’t bother visiting the account at Twitpic.com.  So, with my knowledge of RSS feeds still intact I booted up Yahoo Pipes with it’s clean interface to link my Flickr and Twitpic feeds together.  By aggregating them and filtering them into one feed I could then output that as a widget/script on my website which would make it quicker and easier to get all my photos across to my viewers.

Yahoo Pipes looks complicated when you first play with it because there are so many options in the left hand window with names that sound like they were made up. However clicking on each aforementioned option gives you a brief but concise description in the bottom right informing you what it allows you to do.  Not only does this dramatically decrease the learning curve but also increases experimentation with your feeds, which is great! Anyway I’ll let you play with it at your own speed but all I needed was to add the two feeds together then sort them in terms of date with the newest item first.  There would be no point just aggregating the feeds together with no sorting because then all the resultant feed would show would be all my Flickr pics then all my Twitpic pics which would be rather pointless and tedious for anyone to watch. Here’s how I did it:

1. Open Sources and select “Fetch Feed” and drag onto the canvas section. Do it for however many feeds you want to mix.
2. Open Operators and select “Union” and drag onto the canvas section. This will be the mixing device if you get what I mean.
3. Open Operators and select “Sort” and drag onto the canvas section.
4. Right now presuming you know your feed addresses, copy them into the fetch feed boxes.
5. Now is the time to link them altogether and boy is it easy. Click on the circles underneath the boxes to link the “Fetch Feeds” to the Union. Then link the “Union” to the “Sort” before finally linking the “Sort” to the “Pipe Output
6. The penultimate step is to change the “Sort” settings to item.pubDate in descending order.
7. One final thing is to name the project in the top right and then save it with the button in the top right.

Then all you need to do is test your newly created masterpiece and then think how you want to implement it. Yahoo serve a badge feature which I used to output the new feed as a slideshow type widget on the WordPress blog and seems to do exactly what I want.  There are also other outputs like adding to iGoogle or MyYahoo but with the new RSS feed you can try other third party services (like Widgetbox.com) to create whatever experience you want.  Yahoo Pipes really is an amazing gem if you spend time with it and makes managing numerous feeds a sinch.