View Issue Details

IDProjectCategoryView StatusLast Update
0003322Composrcorepublic2020-03-21 01:14
ReporterChris GrahamAssigned ToChris Graham 
Status resolvedResolutionfixed 
Product Version 
Fixed in Version 
Summary0003322: Cleanup broken URL APIs
DescriptionWe have:
 - test_url [for Comcode]
 - handle_has_checked_recently (uses url_title_cache table)
 - check_url_exists (uses urls_checked table)
 - check in broken URL API

Possibly these can be merged into a single API, using a single table.

It should offer good default functionality. It should be able to store HTTP responses in the table. It should be able to have "only cache if 200 status" functionality (i.e. cache on success, re-try on failure).
TagsRoadmap: v11
Attach Tags
Time estimation (hours)2
Sponsorship open


related to 0003696 assignedChris Graham Google Webmaster Tools errors 


Chris Graham

2019-06-11 01:39

administrator   ~0005963

Last edited: 2020-03-21 00:49

View 2 revisions

"Also having an admin_broken_urls module (v11) is highly questionable. Likely should just roll into Health Check, as a number of separate checks running off of a common API.

Should be different health checks for local URLs, remote URLs, and the other options currently handled in admin_broken_urls."

EDIT: No, because false-positives will happen.

Chris Graham

2020-01-26 23:08

administrator   ~0006310

One thing to note is Cloudflare is sometimes putting out 'challenge pages', with a 403 status code and no way (other than screen-scraping) to know it's because it thinks you may be a bot. Which will be the case for broken URL checks.

So the broken URL checking needs to handle 403 (and probably 401) as not failures. Perhaps make the checks smarter than binary fails/works, with support for a few different response categories, and include this information in the check results.

Issue History

Date Modified Username Field Change
2017-07-23 21:58 Chris Graham New Issue
2017-09-03 22:47 Chris Graham Description Updated View Revisions
2017-09-03 22:48 Chris Graham Description Updated View Revisions
2019-06-11 01:39 Chris Graham Note Added: 0005963
2019-06-17 16:47 Chris Graham Relationship added related to 0003696
2019-06-27 17:58 Chris Graham Tag Attached: Roadmap: v11
2020-01-26 23:08 Chris Graham Note Added: 0006310
2020-03-07 21:20 Chris Graham Assigned To => Chris Graham
2020-03-07 21:20 Chris Graham Status non-assigned => assigned
2020-03-21 00:49 Chris Graham Note Edited: 0005963 View Revisions
2020-03-21 01:14 Chris Graham Status assigned => resolved
2020-03-21 01:14 Chris Graham Resolution open => fixed