View Issue Details

IDProjectCategoryView StatusLast Update
0003032Composrcorepublic2019-10-19 02:02
ReporterChris GrahamAssigned ToChris Graham 
SeverityFeature-request 
Status resolvedResolutionfixed 
Product Version 
Fixed in Version 
Summary0003032: Bulletproof spreadsheet importing
DescriptionYou can currently import CSV files, and it also supports text files with ";" as a delimiter (we don't advertise this).

This is pretty techie for some users.

We should ideally be able to handle any of these scenarios, using a clean API:
1) User accidentally uploads CSV with tab delimiter (TSV) not proper CSV
2) User accidentally uploads CSV with semicolon delimiter not proper CSV
3) User uploads a spreadsheet with no header row in a situation where we don't strictly need one (newsletter raw email list import comes to mind)
4) User uploads a CSV with old-Mac-style line endings (this should not happen in 2016 but some ageing codebases export bizarre stuff; the PHP enable auto_detect_line_endings setting may help)
5) User uploads a spreadsheet with a different character set, esp UTF-16 or ANSI (to solve this we'll probably need to have a character-set selector box and also support BOM markers)
7) Use uploads a very large files that won't fit in memory in one go
Additional InformationCode files that load CSV files at the moment...

adminzone/pages/modules/admin_newsletter.php
sources/hooks/systems/tasks/import_member_csv.php
sources/hooks/systems/tasks/import_newsletter_subscribers.php
sources/hooks/systems/tasks/import_catalogue.php
sources/newsletter.php
pages/modules/recommend.php

We'd want all this to be carefully unit tested.
TagsRoadmap: v11, Type: Cross-cutting feature, Type: Usability (major)
Time estimation (hours)6
Sponsorship open

Relationships

related to 0001908 resolvedChris Graham Support importing of spreadsheets other than CSV 
related to 0003467 resolvedChris Graham Support Unicode BOM markers 

Activities

There are no notes attached to this issue.

Issue History

Date Modified Username Field Change
2017-01-19 22:47 Chris Graham New Issue
2017-01-19 22:51 Chris Graham Tag Attached: Type: Usability (major)
2017-01-26 12:51 Chris Graham Description Updated View Revisions
2017-01-26 12:51 Chris Graham Additional Information Updated View Revisions
2017-05-01 16:05 Chris Graham Tag Attached: Type: Cross-cutting feature
2017-05-01 17:09 Chris Graham Relationship added related to 0001908
2018-02-07 20:44 Chris Graham Relationship added related to 0003467
2018-02-07 20:45 Chris Graham Description Updated View Revisions
2018-02-13 01:23 Chris Graham Time estimation (hours) 12 => 6
2018-02-13 01:23 Chris Graham Description Updated View Revisions
2019-06-27 18:58 Chris Graham Tag Attached: Roadmap: v11
2019-10-19 02:02 Chris Graham Assigned To => Chris Graham
2019-10-19 02:02 Chris Graham Status non-assigned => resolved
2019-10-19 02:02 Chris Graham Resolution open => fixed