View Issue Details

IDProjectCategoryView StatusLast Update
0005063Composrcore_forum_driverspublic2022-11-21 14:09
ReporterPatrick SchmalstigAssigned To 
SeverityFeature-request 
Status non-assignedResolutionopen 
Product Version 
Fixed in Version 
Summary0005063: Use get_table_count_approx for forum drivers / Do not count bots created by phpBB towards number of members
DescriptionCNS uses get_table_count_approx for counting members/topics/posts, because InnoDB is very slow. Ideally forum drivers should use this to.

Also:
phpBB creates a bunch of bot accounts for web crawlers. Do not count these towards total site members in Composr when using the phpBB driver.
TagsNo tags attached.
Time estimation (hours)2
Sponsorship open

Activities

Chris Graham

2022-11-21 03:17

administrator   ~0007670

From our POV it is inefficient to try and work out what are bots, as those no direct way to do it - we'd have to check group membership, meaning a join and less efficient SQL counting mechanism.

I wonder how phpBB itself handles this. Does it not count bots? Does it show a total at all?

Regardless, this isn't a big issue, we just wanted to raise it on the tracker in case people care and to have the discussion.

Patrick Schmalstig

2022-11-21 03:26

administrator   ~0007672

Last edited: 2022-11-21 03:28

View 2 revisions

It is possible to get the member count with 2 queries: first, grab the bot group ID, second, run a COUNT on the users table for all users whose primary group does not match that bot group ID (primary group is on the users table, so no JOIN is needed)

Chris Graham

2022-11-21 08:20

administrator   ~0007677

Is this going to be reliable or does it hard-code a group name that might be edited / vary by language?

Also in the ideal world we'd be using get_table_count_approx for the member/topic/post count queries in the forum drivers, as counting full table rows on InnoDB is very slow. We're currently only doing that for CNS. If we do that, we either need to not do the filtering or actually have a third query for the number of bots to subtract (rather than doing it as part of a WHERE clause). I'll add that to this issue.

Patrick Schmalstig

2022-11-21 14:09

administrator   ~0007680

It looks like it would be reliable; the group name is BOTS and cannot be edited in the phpBB interface. I ran a Spanish install and it was still BOTS in the database (the other group names were also the same).

Issue History

Date Modified Username Field Change
2022-11-19 18:11 Patrick Schmalstig New Issue
2022-11-21 03:15 Chris Graham Time estimation (hours) => 1
2022-11-21 03:16 Chris Graham Category General => core_forum_drivers
2022-11-21 03:17 Chris Graham Note Added: 0007670
2022-11-21 03:26 Patrick Schmalstig Note Added: 0007672
2022-11-21 03:28 Patrick Schmalstig Note Edited: 0007672 View Revisions
2022-11-21 08:20 Chris Graham Note Added: 0007677
2022-11-21 08:21 Chris Graham Summary Do not count bots created by phpBB towards number of members => Use get_table_count_approx for forum drivers / Do not count bots created by phpBB towards number of members
2022-11-21 08:21 Chris Graham Description Updated View Revisions
2022-11-21 08:21 Chris Graham Time estimation (hours) 1 => 2
2022-11-21 14:09 Patrick Schmalstig Note Added: 0007680