Question

At the moment I’m using Memcache to store my php pdo mysql queries and it’s working just as intended.

However I would like to create a single page sitemap with ~50,000 urls and think I may need a different way to cache this query. Ideally I’d only want to perform the query once per day and think that keeping 50,000 ids and titles in memcache for that amount of time would be a complete waste of space?

Would it be better to gzip the files into some sort of temporary .txt file? or a different method completely?

Answer 1

I agree with Marc’s comments. Generate the file once per day, perhaps with a cron job.

I refer you to this statement about sitemaps, which you should consider:

Sitemap files have a limit of 50,000 URLs and 10 megabytes per
sitemap. Sitemaps can be compressed using gzip, reducing bandwidth
consumption.

Currently, I’m generating my sitemap dynamically, but switching it over to a cron job is on my to do list.