Speeding up PHP RSS retrieval

blue_harvester · 4 Feb 2007 at 22:44

I'm building a simple one page website that displays the top 10 stories of 9 different RSS feeds. I'm using PHP PEAR to retrieve the RSS feeds. When you first visit the site it takes approx 15 second to retrieve all feeds. This is clearly far too long to wait and therefore I need a way to have this data readily available when someone visits the site.

Any suggestions? Is it possible to have a PHP script that runs every 1 minute that will go off and retrieves the RSS feeds and saves them to an XML on the server. Then I could just grab the XML content with JavaScript?

Beansprout · 4 Feb 2007 at 22:47

Setup a script to do that, and use cron to run the script every X minutes

blue_harvester · 4 Feb 2007 at 23:09

Beansprout said:
Setup a script to do that, and use cron to run the script every X minutes

Could you please elaborate, I'm not too familiar with Unix. I assume that I'll be able to use this on my Linux hosting account? So what you're saying is that I can setup a script that executes the php that retrieves the RSS scripts so that the XML will be updated every x minutes?

Beansprout · 5 Feb 2007 at 00:30

Write the script to do the XML as you suggested in your post, then ask your hosting provider if you have access to 'cron jobs' - these are like scheduled tasks basically, so you can set one up to call the script every now and then (Say every 5min, or something)

Dj_Jestar · 5 Feb 2007 at 00:57

Use caching.

Code:

<?php

if ((time() - filemtime($rss_file)) > 3600) regenerateRssFile();

echo file_get_contents($rss_file);

?>

Beansprout · 5 Feb 2007 at 01:03

Dj_Jestar said:
Use caching.

Code:

<?php if ((time() - filemtime($rss_file)) > 3600) regenerateRssFile(); echo file_get_contents($rss_file); ?>

But then some visitors will still have to wait a while

Dj_Jestar · 5 Feb 2007 at 01:34

If it's taking "too long" to retrieve only 10 items, something else is wrong.

besides which.. 15 seconds once, every hour, for one person, is not the end of the world.

blue_harvester · 5 Feb 2007 at 09:06

Dj_Jestar said:
If it's taking "too long" to retrieve only 10 items, something else is wrong.

besides which.. 15 seconds once, every hour, for one person, is not the end of the world.

The reason it takes so long is because a number of the RSS feeds take a while to be retieved from the remote site; I've tried this in a browser by loading just the RSS XML and it takes maybe 3 or 4 seconds. When all 9 RSS feeds are being retrieved at once then this is the problem.

Also, the target audience for this site will not be prepared to wait 15 seconds when they first load the page. So therefore the information needs to be readily available.

isset · 5 Feb 2007 at 09:30

Beansprout said:
Setup a script to do that, and use cron to run the script every X minutes

edit: ignore. i can't read.

Slogan · 21 Jul 2008 at 17:28

Thought I'd bump this rather than make a new topic.

I have a forum RSS feed and I want to insert it into my sidebar (php I'm guessing).

I did use a conversion for this a while ago (RSSinclude) but I since changed my domain and when I went to re-do this with the new URL, it slowed the whole site down and the sidebar took ages to load.

So today I've sort of been looking around places for alternatives and there really isn't a lot about. I've installed MagpieRSS along with Smarty which seems to be along the right lines of what I need but it generates an ugly results page with cell borders as red, date text as black and links as blue.

I just want the links (forum thread titles) and want them to be black but I can't seem to find information on how to change the style or formatting. Anyone got any links or ideas?

Thanks.

Template file is:

Code:

{* very simple template for a table with RSS items *}
<table>
{* Show the feed title as a header in the first row of the table *}
<tr><th style="font-size: 24px; border: 1px solid black;">{$feedtitle}</th></tr>
{* loop through all items and add date, link, title and description in the follwing section *}
{section name=x loop=$items}
<tr><td class="rssitem" style="border: none;">
<strong>{$items[x].pubdate}: <a href="{$items[x].link}" target="_blank">{$items[x].title}</a></strong><br>
{$items[x].description}
</td></tr>
{/section}
</table>
{* That's it. Table can be formatted using some CSS classes *}

Managed to prise away the red border which was obvious now that I look at it. Just going through it but if someone has a better grasp of it then feel free to post.

Do I tear out:

Code:

{$items[x].pubdate}:

?

edit --

Okay the answer to that was yes. I'm becoming a pro...

Anyone got any idea how to make the links (.link) black now instead of the standard blue?

Dj_Jestar · 21 Jul 2008 at 17:47

blue_harvester said:
The reason it takes so long is because a number of the RSS feeds take a while to be retieved from the remote site; I've tried this in a browser by loading just the RSS XML and it takes maybe 3 or 4 seconds. When all 9 RSS feeds are being retrieved at once then this is the problem.

Also, the target audience for this site will not be prepared to wait 15 seconds when they first load the page. So therefore the information needs to be readily available.

Only one person would need wait 15seconds, not everyone, and it would be whomever is unlucky enough to click the link when the old cache expires after an hour.