Scheduling Fetching of Feeds (Imports) for Better Performance
Whether you are already encountering the below issues or not, setting up different schedules for multiple feed sources will help you to ensure that your site runs smoothly. It will also minimize the potential for your WordPress cron system to get stuck.
Some of the possible issues being caused by having too many feed sources trying to fetch (import) new content at the same time could include:
- Slower load speed
- Users being locked out of the site
- Your site timing out when trying to import from a large number of feed sources at once
WP RSS Aggregator currently has two separate methods of updating feed sources:
The first is the global interval. This is the option you set in your general settings (RSS Aggregator > Settings). Any feed sources that do not have their own update interval will be updated on this schedule.
- The second is each feed source's own interval, which you can set/modify through the Update Interval option (in the Feed Processing meta box, on the right-hand side of the Add/Edit Feed Source screen).
Thus, when a large number of feed sources are all using the global interval when the time comes for the plugin to update the feed sources, the site may crash or hang due to a large number of update operations happening simultaneously.
The solution is to schedule the updates in one of two ways.
Individual Updating
Set all feed sources’ Update Interval to be anything other than “Default”. For instance, some of them would be set to update every hour, some of them once a day, some others every 2 hours, and so on.
You can cleverly set the interval according to the feed source. If a feed source only has 1 new post per day, you don’t need to update it every 2 hours. Setting it to update daily, starting sometime after the post is normally published would work great.
In order to do that, you would need to edit the feed source around that said time, set the update interval to be 1 day, and Update the feed source with the new option.
Batch Updating
We only suggest using this method if you need all of your feed sources updated using the same interval. For instance, all feed sources update hourly. If you can do without this, please do, as it will ensure that your site won't experience the above-mentioned issues.
- Pause all your feed sources.
- Edit each of your feed sources, and set their Update Interval to your liking.
- Let’s say the current time is 1 PM, and we are setting the update interval to 1 hour.
- Activate the first batch of feed sources. Try to keep the number of feed sources in each batch to a maximum of 15. You can do this by bulk selecting the feed sources and using the Activate bulk action from the Feed Sources list.
- These feed sources will start updating hourly, on the hour.
- Wait for some time, for example, 20 minutes, and activate the next batch of 15 feed sources.
- Alternatively, set the next batch of feeds to activate at a future time, so you won’t have to wait. You can do this from the Feed Processing meta box in the Add/Edit Feed Source page.
- These will start updating hourly, in the 20th minute of each hour.
- Repeat the previous step. The next batch of feed sources will start updating hourly, in the 40th minute of each hour.
- Keep repeating this process until all your feed sources are activated. The more sources you have, the more you may have to shorten the time between each bulk activation.
This will allow you to have your feed sources update using the same interval, but update in batches, at different times.
Also, keep your server’s workload in mind when dealing with large operations, not just with WP RSS Aggregator. It can help you avoid such issues, and maybe even improve your site’s performance.
Feed Source Update Interval
The individual feed source’s update interval can be found in the meta box titled “Feed Processing”, as seen below.
General Update Interval
The plugin’s general, or default, update interval can be found in RSS Aggregator > Settings > General > Feed Processing Interval, as seen below.
