View Issue Details

IDProjectCategoryView StatusLast Update
0003958Composrcorepublic2019-11-19 01:19
ReporterChris GrahamAssigned To 
Status non-assignedResolutionopen 
Product Version 
Fixed in Version 
Summary0003958: Opportunistic scheduler
DescriptionSome background Cron hooks are not very time critical, but may have a performance impact. Ideally we would run these hooks only when the server is under low load.

Code in a detector function to find if the server is under 'low load', based on I/O load, CPU usage (uptime command on Linux), and memory usage (configurable thresholds).
This check would be run before every Cron hook that is algorithmically determined as 'optional'.

Make the scheduler (Composr CRON system) use this to determine whether some low priority hooks should get done. For example, updating the sitemap. CRON hooks would specify:
 - a "frequency" (currently this logic is hard-coded in v10, and already implemented as the 'minutes_between_runs' setting in v11)
 - a "fluff" factor (to determine how much things should be allowed to be late, e.g. 1.6x -- tasks being deferred if not in a low load state)
 - a "priority" (i.e. tasks would run in priority order [well actually anything already late past it's fluff factor would be elevated to high priority])
...and the system would automatically launch CRON hooks accordingly.

Hooks should be able to keep calling back and checking to see if they should keep going. Not all hooks would implement that, but particularly long running low priority re-entrant ones (background caching/indexing) would.
TagsocProducts client-work (likely), Type: Performance
Attach Tags
Time estimation (hours)4
Sponsorship open


related to 0003501 non-assigned Poor-mans-Cron 
related to 0002236 closedChris Graham Opportunistic scheduler and background caching 
related to 0003959 non-assigned Cache warm up via Cron 
related to 0003185 non-assigned Pagination indexing table specifically for forum topics 
related to 0003288 resolvedChris Graham Major overhaul of fulltext search support, the "Composr fast custom index" 


There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2019-11-19 01:12 Chris Graham New Issue
2019-11-19 01:12 Chris Graham Tag Attached: ocProducts client-work (likely)
2019-11-19 01:12 Chris Graham Tag Attached: Type: Performance
2019-11-19 01:13 Chris Graham Relationship added related to 0002236
2019-11-19 01:13 Chris Graham Relationship added related to 0003501
2019-11-19 01:14 Chris Graham Relationship added related to 0003959
2019-11-19 01:16 Chris Graham Relationship added related to 0003185
2019-11-19 01:17 Chris Graham Relationship added related to 0003288
2019-11-19 01:19 Chris Graham Description Updated View Revisions