View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0003093 | Composr | galleries | public | 2017-02-27 18:46 | 2020-03-07 01:30 |
| Reporter | Chris Graham | Assigned To | Patrick Schmalstig | ||
| Severity | Feature-request | ||||
| Status | resolved | Resolution | fixed | ||
| Product Version | |||||
| Fixed in Version | |||||
| Summary | 0003093: Move gallery import to a task | ||||
| Description | If you are uploading more than say 70 large images to a gallery at once, it may timeout on some servers. (The actual figure varies, I am seeing a server doing a CGI timeout after 140 seconds, with 3 second processing time per image) We could therefore do the import in a task. Maybe doing that based on the number of images being imported. | ||||
| Additional Information | A workaround is to pre-size the images down to the maximum image size. This reduces the processing time per image, roughly double the limit. | ||||
| Tags | Roadmap: v11, Type: Performance | ||||
| Time estimation (hours) | 3 | ||||
| Sponsorship open | |||||
|
|
Proposed solution: Your scenario of a 140-second timeout with 3 seconds per image yields a maximum processing capability of only 46 images before timeout. Therefore, I could make import immediate if 25 images or under. Otherwise, schedule a task that processes the images in batches of 25 (or a configurable option value). Or, sense server timeout and whatever images have not yet been processed, re-trigger the task with the remaining images. |
|
|
The problem is it really depends what images are imported. People often nowadays don't think about it when importing enormous megapixel images from smartphones. I don't think it's reasonable for us to try and compute it. How about this... 1) Rework the code so it can be run both as a task and from the module. 2) Make it so that code can start importing from an offset into the set of images being imported (needs to consider that there may both be multiple files and archive files) 3) Import immediately, but if (time() - $_SERVER['REQUEST_TIME'] > 20) create the task, called with the offset |
|
|
For the record, we decided to ignore the issue about tasks being slow and hence not wanting small import jobs to run as tasks. Instead, we're implement a new way of Cron executing whereby it stays resident in a loop, so tasks can be identified and finished very quickly. Additionally, some kind of console for monitoring task progress. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2017-02-27 18:46 | Chris Graham | New Issue | |
| 2019-02-27 18:09 | Chris Graham | Tag Attached: Type: Performance | |
| 2019-06-27 17:58 | Chris Graham | Tag Attached: Roadmap: v11 | |
| 2019-07-12 21:21 | Chris Graham | Assigned To | => Patrick Schmalstig |
| 2019-07-12 21:21 | Chris Graham | Status | non-assigned => assigned |
| 2019-12-24 17:22 | Patrick Schmalstig | Note Added: 0006248 | |
| 2019-12-24 18:30 | Chris Graham | Note Added: 0006257 | |
| 2020-03-05 21:05 | Patrick Schmalstig | Status | assigned => resolved |
| 2020-03-05 21:05 | Patrick Schmalstig | Resolution | open => fixed |
| 2020-03-07 01:30 | Guest | Note Added: 0006465 |
