View Issue Details

IDProjectCategoryView StatusLast Update
0003826Composrstatspublic2020-01-06 16:48
ReporterChris GrahamAssigned ToChris Graham 
Severityfeature 
Status closedResolutionno change required 
Product Version 
Fixed in Version 
Summary0003826: Move stats CSV download to a background task
DescriptionCurrently the stats download runs in real-time, but this may timeout, so should be done in a background task.
TagsRoadmap: v11, Type: Performance
Attach Tags
Time estimation (hours)3
Sponsorship open

Relationships

related to 0003955 resolvedChris Graham Move subscriber export code to a task 

Activities

Chris Graham

2019-10-18 20:48

administrator   ~0006123

Very large sites are unlikely to use our inbuilt stats system, but rather something like Google Analytics - so it's not a big problem.

Chris Graham

2019-11-26 22:08

administrator   ~0006183

Patrick - look at sources/hooks/systems/tasks to see examples of current task hooks.

Tasks allow items to be queued for execution via Cron, not subject to timeouts, and run in an orderly manner so as to not over-stress the server.

A task is initiated using code like (code from cms_blogs.php)...

            require_code('tasks');
            return call_user_func_array__long_task(do_lang('IMPORT_WORDPRESS'), $this->title, 'import_wordpress', array($is_validated, $download_images, $to_own_account, $import_blog_comments, $import_to_blog, $import_wordpress_users));

Where the array is whatever options need to be passed to the task.

By default a task will run immediately. That's fine for testing them.

admin_stats.php is very old code and not ideal in how it is structured. You may choose to move a lot of it out to new functions that can serve data both for the task hook and for the admin_stats module.

Chris Graham

2019-12-24 20:18

administrator   ~0006263

Reassigned to me as I'm currently considering reimplementing the stats code for v11. The Health Check addon needs to be able to consume stats data, to be able to warn of concerning dips in stats - so changes beyond just merely moving code around is needed.

Chris Graham

2020-01-06 16:48

administrator   ~0006270

In v11 stats is being completely rewritten. There's now a pre-processing step, which happens in Cron. This means things like CSV download can happen in real-time as it's going from pre-processed data (much quicker to prepare a report).

Issue History

Date Modified Username Field Change
2019-06-21 19:17 Chris Graham New Issue
2019-06-21 19:18 Chris Graham Tag Attached: Type: Performance
2019-10-18 20:48 Chris Graham Note Added: 0006123
2019-11-16 03:44 Chris Graham Tag Attached: Roadmap: v11
2019-11-16 03:46 Chris Graham Relationship added related to 0003955
2019-11-26 22:06 Chris Graham Assigned To => Patrick Schmalstig
2019-11-26 22:06 Chris Graham Status non-assigned => assigned
2019-11-26 22:08 Chris Graham Note Added: 0006183
2019-12-24 20:17 Chris Graham Assigned To Patrick Schmalstig => Chris Graham
2019-12-24 20:18 Chris Graham Note Added: 0006263
2020-01-06 16:48 Chris Graham Status assigned => closed
2020-01-06 16:48 Chris Graham Resolution open => no change required
2020-01-06 16:48 Chris Graham Note Added: 0006270