Merging RSS Feeds With SimplePie

Have you played around with SimplePie? If not, start experimenting. SimplePie is an amazingly useful file you include in your PHP scripts so you can parse RSS feeds. It’s fast, it’s powerful, it’s easy to use, and there’s a lot you can do with it.

In addition to the ordinary stuff, like displaying feed content on your website, you can also utilize SimplePie for more interesting things….like combining RSS feeds. Why would you want to do that? Well, some of us have multiple blogs. Using SimplePie you can create a feed that aggregates all of your blogs’ feeds. Blog networks may find this technique useful as well.

How do you merge feeds? It’s actually quite simple. Your script should do the following:

  • Echo the beginning of the XML (the character encoding must be UTF-8)
  • Load simplepie.inc
  • Store all the original feeds in an array
  • Initialize SimplePie
  • Loop around echoing the aggregated RSS items
  • Echo the end of the XML

Not too complex, eh? Let’s get coding!

First you need to output the beginning of the XML structure

<?php echo ‘<?xml version=”1.0″ encoding=”UTF-8″?>’; ?>
<rss version=”2.00″>
<channel>
<title>Aggregated Feed Demo</title>
<link>http://www.webmaster-source.com</link>
<description>
A demo of SimplePie’s feed-merging capabilities.
</description>
<language>en-us</language>

Just basic XML. Note that you need to use UTF-8 encoding, otherwise you could have some oddities in the feed. SimplePie prefers UTF-8, and can be a little buggy with other encodings.

Next, the script takes care of some SimplePie stuff and loads the RSS feeds.

<?php
include_once(‘./lib/php/simplepie.inc’); // Include SimplePie

$feed = new SimplePie(); // Create a new instance of SimplePie

// Load the feeds
$feed->set_feed_url(array(
‘http://feeds.feedburner.com/Webmaster-source’,
‘http://feeds.feedburner.com/ProbloggerHelpingBloggersEarnMoney’
));

$feed->set_cache_duration (600); // Set the cache time

$feed->enable_xml_dump(isset($_GET['xmldump']) ? true : false);

$success = $feed->init(); // Initialize SimplePie

$feed->handle_content_type(); // Take care of the character encoding
?>

The only things you really need to change above are the feed URLs and the path to simplepie.inc. Now for the fun part. Let’s output ten RSS items, shall we?

<?php if ($success) {
$itemlimit=0;
foreach($feed->get_items() as $item) {
if ($itemlimit==10) { break; }
?>

<item>
<title><?php echo $item->get_title(); ?></title>
<link><?php echo $item->get_permalink(); ?></link>
<description>
<?php echo $item->get_description(); ?>
</description>
</item>

<?
$itemlimit = $itemlimit + 1;
}
}
?>

</channel>
</rss>

That was easy, wasn’t it? Just save it as rss_all.php, or something similar, and upload it to your web server. Try it out. Note that it’s a good idea to use Feedburner so your server doesn’t have to process everything when a user’s RSS reader requests the feed. The SimplePie website has a complete API reference you may find useful. With a couple minutes of coding, you can easily add a <pubDate> tag to the generated feed. I highly recommend doing so, as it will make it easier for others to mix your aggregated feed with some of theirs.

Want to see this script in action? Take a look at this feed. It has the ten most recent items from the NTugo Blogs feeds

Download TXT file (copy-pasting code from blog post may not work).

  • http://heartdaughter.com/blogs/elijah/ ladynada

    Good job! Thank you for sharing this. I am working with simplepie now. Their site is down and I needed some code!

    nada

  • http://186kpersecond.com/mashup 186

    I have been looking for and trying to do exactly what you describe but with no luck.

    I used your sample and no luck. I changed some of the syntax errors and still wont fire.

    Thank you for posting this.

  • http://www.webmaster-source.com Matt

    Dang, WordPress is changing my code again…

    It’s changing strait quotes into curly quotes, and things like that. I’ll put-up a downloadable txt file to straighten it out.

  • http://www.webmaster-source.com Matt

    There. You can download the code here. There may be some differences in the coding style, but it’s essentially the same. I cleaned-up my code when I published it in the blog post, but I didn’t have time to do it in the txt file. It works though. Enjoy!

  • Richard

    I’m getting posts shuffled. is there a way to force a sort by post time?

  • http://www.webmaster-source.com Matt

    Um… Are the feeds you are merging all RSS 2.0? SimplePie seems to have a bug where they don’t go in the right order if they don’t have the pubDate tag in the feeds.

  • Richard

    It looks like there were incompatible date formats – one atom, one RSS 2.0.

  • http://www.webmaster-source.com Matt

    Can you give me the URLS of the feeds? I’ll look into it.

  • http://www.hachemuda.com Guillermo

    Hi, congratulations and thanks for this great tutorial.
    I have used your code to create a feed merging all contributor feeds. Here you can see it working:
    The Planet: http://www.planetubuntu.es
    The feed: http://www.planetubuntu.es/rss.php

    It’s working fine, but I have a problem with the dates an hours which are shown in the feed reader. In my feed reader, I reload my subscription to PlanetUbuntu and it puts all the new articles with the same date. I have an screenshot: Feed PlanetUbuntu Screenshot.

    Can you help me, please?
    Thank you very much.

  • http://www.webmaster-source.com Matt

    Viewing the source of your merged feed (http://www.planetubuntu.es/rss.php), I don’t see any pubDate tags.

  • venkat

    hi… where and how to create smallpie.inc file.. please help me to get that file

  • http://www.webmaster-source.com Matt

    You can find it at http://www.simplepie.org.

  • fabme

    I would like to know where i can find informations about feeds and the law ?
    can i do a website and publish any feeds on it ??

  • http://www.webmaster-source.com Matt

    Well, you’ve touched upon a complicated subject, fabme. If you display feeds in a manner similar to popurls.com (linked titles, maybe a short excerpt), then you’re okay. Most people would not want you to reproduce entire articles though.

  • http://photo.busdraghi.net fabusdr

    hello
    thank you for sharing this, it is really useful for me.

    I just suggest you to use simplepie core plugin, instead to load manually the library. as they write on their site:

    http://simplepie.org/wiki/plug.....lepie_core

    Don’t load SimplePie manually by including/requiring simplepie.inc or idna.class.php. Instead, check to see if the SimplePie class is defined. If it is, you’ll know that SimplePie is available to be used.

    Thanks again

    f

  • http://www.webmaster-source.com Matt

    fabusdr, thanks for the info. I was unaware of this option. I’ll have to remember it for future use.

    However, it probably wouldn’t work well in this situation. This tutorial is made to work outside of WordPress, as a standalone script.

  • http://186kpersecond.com 186

    Hello again.Is it me or is there a problem with parsing “Feedburner” feeds?Anytime I include one it does not show and it returns various results with the number of feeds in the output.

  • http://www.webmaster-source.com Matt

    That’s strange. I never have any problem with FeedBurner. Perhaps it has to do with the specific feeds, not Feedburner itself? Try parsing my FeedBurner feed: http://feeds.feedburner.com/Webmaster-source and see if it does anything.

  • Sam England

    The script wotks great with these feeds instance:
    http://www.timesonline.co.uk/t.....siness.xml‘,
    http://www.timesonline.co.uk/t.....s/tech.xml

    But not these

    http://feeds.reuters.com/reuters/technologyNews‘,
    http://feeds.reuters.com/reuters/internetNews

    or these

    http://timesofindia.indiatimes.....efault.cms‘,
    http://timesofindia.indiatimes.....479906.cms

    It seems the script will work with feeds with .rss, .xml, extensions

    The script returns nothing when used with scripts with no extensions
    (like reuters use above), or custom extensions like .cms (like
    indiatimes uses above)

    Is there a simple edit to my script, or simplepie.inc that might fix
    this? In simplepie.inc line 12634 I can see this in reference .rss,
    .xml etc extensions but im not sure if it is anything to do with it:

    if (in_array(strtolower(strrchr($value, ‘.’)), array(‘.rss’,
    ‘.rdf’, ‘.atom’, ‘.xml’)))

    Thank you in advance for any help

  • http://www.webmaster-source.com Matt

    @Sam, I’ve heard of people having the same problem as you, but have never experienced it myself. I would make sure you have the latest stable version of SimplePie, and if that doesn’t help I’d post a help request on the SimplePie support group here: http://tech.groups.yahoo.com/g.....e-support/

    I haven’t been able to solve the problem for people who have had it before, but I suspect if you post on the SimplePie forum, and include the code you’re using, they will have a solution.

  • Pingback: links for 2008-06-22 « marka buku

  • http://www.deuts.net/ deuts

    In the text file, “php” doesn’t really appear after the “<?” ?

  • http://www.deuts.net/ deuts

    with the $item->get_description(), it returns a summary feed. I want to show the whole post. Could you please tell us how to do that? the get_content doesn’t seem to work. thanks.

  • http://www.webmaster-source.com Matt

    @deuts To answer your first question, “<?” is a shortcut for “<?php”. I was a little lazy when I typed out the version seen in the text. It should work on most systems.

    About $item->get_description()… Make sure the input feed is full. If it’s a summary feed, you won’t be able to get a full feed out of it. get_content() is what you need though if you want to output a full feed. Make sure you’re using the proper syntax of echo $item->get_content();

  • http://www.deuts.net/ deuts

    Hey Matt, thanks for that response. But I just can’t seem to make it to work (the full feed). I was just trying to consolidate the feeds from my blog (WordPress) and photoblog (pixelpost). The pictures in pixelpost was doing fine, but the feeds from WordPress (www.deuts.net) were truncated. I’m nearly giving up….:D

  • http://www.webmaster-source.com Matt

    @deuts. Have you tried posting on the official SimplePie support group? http://tech.groups.yahoo.com/g.....e-support/ Try posting your full code their, along with a detailed description of your problem. They normally find a workable fix pretty fast.

  • http://www.aizokroon.nl Frog

    I have been trying to add multiple feeds, but just entering another feed doesn’t work.I have a probleem too with the feed url “feed://www.macfreak.nl/readnews.php?action=rss“. All the other feed url’s are working, but that one doesn’t.My feed url is “http://www.aizokroon.nl/widgets/feed.php“.

  • http://www.webmaster-source.com Matt

    @Frog Make sure you use the correct syntax. Compare it to the example above and make sure you have the array, the commas, and the quote marks.

  • http://www.aizokroon.nl Frog

    Oeps, I had forgotten the commas (is this correct english).I now have two problems left.One is that the feed doesn’t work anymore when I add feed://www.macfreak.nl/readnews.php?action=rss. My other problem is that I don’t get more than 10 posts in the feed. Changing $itemlimit==10 doesn’t solve the problem.Do you know how I can solve both problems?

  • http://www.aizokroon.nl Frog

    Oké, I now have 50 posts, but I have still the problem with the above feed. I too have a probleem with the pubDate tag. When I add it my feed only shows 1 post, instead of 50 posts.

    Do you have an idea how I can solve the problems?

  • Pingback: links for 2008-07-24 » Oliver Willis

  • http://www.aizokroon.nl Frog

    Nobody haves an idea?

  • http://www.webmaster-source.com Matt

    @Frog. I’m not sure what your problem is exactly. I’d recommend trying the official SimplePie support group http://tech.groups.yahoo.com/g.....e-support/

  • http://www.aizokroon.nl Frog

    Ok, I will try it there.Can you maby post an example of the pubDate tag in this script. Maby i’m just trying it the wrong way.

  • http://www.TechHairBall.com TechHairBall

    This is a wonderful tool. I used it on my blog and now working on using it for my website. Great job guys.

  • http://whatsonnews.com whatsonnews

    Great job buddy. Simple pie is just awesome! I find yahoo pipes a bit confusing though.

  • Pingback: Merging RSS Feeds With SimplePie | Webmaster-Source – auguag

  • http://imzhcpjb.com Kazeljap

    Hi!chvi! http://ryntznrs.com uyyum cdqpz http://gweocgdv.com yglig ijpze

  • Caleb

    how would i be able to to filter the feeds based on year? (eg. only show the feeds from 2007)

  • Gianfranco

    Is it possible to setup SimplePie to have a website displaying multiples feeds, but :

    when you click on a feed "item" (permalink), instead of going to the item's full article on the feed's website, you dispaly the article on your own website, with your style (and of course you provide a link to the original source / item's article on feed's website).

    Kin dof, instead of that:
    list of feeds > list of items > original article on feed website.

    That:
    list of feeds > list of items > content > (link to original article on feed website).

    Thanks in advance for enlighting me.

  • http://intensedebate.com/people/redwall_hp redwall_hp

    It's *possible*. You could change the link that would go to the source to yoursite.com/display_permalink.php?q=[permalink_URL] then have that page parse the feed again and display the content that matches the permalink. Or something like that…

    Note that you must display summaries of content only. Displaying the full content on your site would be copyright infringement, and illegal in most countries. (And Google also will penalize sites that have the same content as another.)

  • Gianfranco

    Thanks for the answear, redall_hp.
    Of course it should display summary and not the whole content. Thing is, I am supposed to build something for a client and I wanted to make sure that if they need something like that, I don't go: oh, I don't actually know, I'll check it out.

    Anyway, I will test your code later this week to see if and how it does the job.

    Mmm, should I just replace:
    <a href="<?php echo $item->get_permalink(); ?>">

    with:

    <a href="http://www.mysite.com/display_.....t;&gt;

    I don't think I get it right…

    Thanks a lot…

  • http://intensedebate.com/people/redwall_hp redwall_hp

    That would be the start. Then you would have to create a page called display_permalink.php that would have some SimplePie logic in it. You're probably better off asking on the SimplePie support forum for the specifics http://tech.groups.yahoo.com/group/simplepie-supp… They're the real experts. :)

  • http://www.organicjar.com jason cairns

    hi, I want to change the url link in the title of my rss feed. See http://www.healthhive.com/twitter Each feed has a title that goes to that twitter id, but I'm going to use yahoo pipes to remove the twitter username from the rss feed, when I do that, the url title nolonger goes to the twitter name, but to a yahoo pipes website. I'm so lost on how to fix this, there must be some easy way to change the link of the rss title of the feed in simple pie! Please Help

  • Pingback: 14 Tutorials to Get You into SimplePie - The Designed

  • Pingback: benjamin junior » Blog Archive » Bookmarks em August 12th

  • http://www.couct.cn Ryan.J

    this post has send a years, but very useful, i find out the A Collection of Beginner Simplepie Tutorials, maybe can helpful somebody like me for a Beginner
    http://www.devirtuoso.com/2009/07/a-collection-of

  • Designs Edge

    I am testing your script, and love it thus far. I would like the ability to show more than 10 items, I have changed the variable to:
    php foreach($feed->get_items() as $item):
    if ($itemlimit==20) { break; }

    Any idea why I am still getting 10? I have adjusted the cache value as well.

    • http://www.webmaster-source.com Matt

      First, make sure that the feeds have more than ten items altogether. :)

      Also, make sure you delete the files in the cache.

  • ken

    Thank you so much for this guide. I really appreciate you posting it.
    I would like to also add these two items to each of the feed: a category and image (link) unique to each single feed. I will have the information stored somewhere and I just want the values to be assigned to each feed line as the code is looping through the feeds.
    Does anyone know how I can do that? I will really appreciate it.
    Jack.

    • http://www.webmaster-source.com Matt

      You could create a multidimensional array containing the feed URLs and your other data, which you could probably match up with the items somehow?

      • ken

        how do you do that.
        I have researched and I can’t seem to find a way to.
        Any help will be much appreciated.

  • http://www.mossackanme.web.id/feed/ Mossack Anme

    Why my feed cant load the description and date? See it through my name.

  • Linda Harrison

    Hi,

    I tried to test this out on my server, and it doesn’t seem to be working. Can you give me any suggestions? This is where I did it, I copied everything precisely and only changed where the SimplePie.inc file was included http://lindaharrisonmusic.com/rss_all.php
    Any help would be greatly appreciated!

    Linda x

  • Bearman

    maybe geshi would help for code on wordpress ;)http://wordpress.org/extend/plugins/scgeshi/

    and for validating the generated rss-feeds: http://www.feedvalidator.org

    greetz
    Bearman

    PS: (it seems there is a broken JS …)

  • http://www.widgetsandburritos.com David S.

    I did something similar recently, but ended up writing the class myself. It does a lot of the same stuff as SimplePie, but also sorts by pubDate.

    If anyone is interested in seeing what I did, I blogged about it here:
    http://www.widgetsandburritos......php-cache/

  • Jan

    Hello! Is it possible to add to each RSS item the title of the website or feed?

    I mean, if I have 2 feeds:
    http://feeds.feedburner.com/Webmaster-source
    http://feeds.feedburner.com/Pr.....sEarnMoney

    I would like to have the result like this:
    WebmasterSource: Item title nr. 1
    WebmasterSource: Item title nr. 2
    ProBlogger: Item title nr. 3

  • http://www.muschamp.ca/ Muskie

    I think what Jan wants to do is totally doable, you need to get items associated feed and attach that title, something like:

    $item->get_feed()->get_title() . ‘:’ . $item->get_title()

    I’m going to try this code out, but what I wanted to know about was thumbnails associated with the feed, some people are big on thumbnails and I wanted to preserve them…

  • http://www.muschamp.ca/ Muskie

    I don’t think this is working with the latest version of SimplePie. Even fixing the line about forcing XML to:

    $feed->enable_xml_dump(true);

    It generates an error about header rewriting:

    Warning: Cannot modify header information – headers already sent by (output started at /home/muskie/domains/muschamp.ca/public_html/news/mergedFeed.php:1) in /home/muskie/domains/muschamp.ca/public_html/news/php/simplepie.inc on line 1751

  • http://www.muschamp.ca/ Muskie

    I got it fixed, you want to remove the line:

    <?php echo ‘’; ?>

    And move the rest of the first section after you call:

    $feed->handle_content_type(‘text/xml’);

    I also specify XML here and I changed this line too:

    $feed->enable_xml_dump(true);