View Issue Details

IDProjectCategoryView StatusLast Update
0001389Composrcorepublic2015-07-14 23:45
ReporterChris GrahamAssigned ToChris Graham 
SeverityFeature-request 
Status resolvedResolutionfixed 
Product Version 
Fixed in Version 
Summary0001389: XML Sitemap index
DescriptionCurrently Composr builds a single sitemap nightly. For a large website, this is resource-intensive and risks going over the 50,000 sitemap URL/10MB limit.

Ideally we would support the sitemap API putting out everything since the last "archival date", and then iteratively updating the sitemap index file to include the refreshed reference to the latest sitemap. It would have to calculate the archival date in a smart way, noting when it approaches a certain limit (e.g. 5000 URLs per sitemap).
Additional InformationOld sitemaps would contain URLs duplicated in newer sitemaps, if the article had been refreshed. This is not a serious problem though.
TagsNo tags attached.
Time estimation (hours)10
Sponsorship open

Relationships

related to 0000142 resolvedChris Graham Merged sitemap API 

Activities

Chris Graham

2015-07-14 23:45

administrator   ~0002954

Resolved in https://github.com/ocproducts/composr/commit/29c9de41415bd5c07bd75377da7dc2ffa0d2eebc

Implementation differs to description. Actually works much better, this is very efficient (can scale to millions of URLs), and can handle deletions of URLs from the sitemap.

Issue History

Date Modified Username Field Change