Protecting the data directory with .htaccess?
Standard member
google console is complaining about Server error (5xx)
Hello,Received a "New Coverage issue detected for site https://nandalow.com/" warning email from google search console. Apparently they are scanning my https://nandalow.com/data/ directory and getting some 5xx type errors…..
For example:
https://compo.sr/data/notifications.php
google is complaining about the data/notifications.php…. Would you recommend we add to the disallow list on the robots.txt file?
The robots.txt does not specifically block the /data/ folder (I dont think):
https://compo.sr/robots.txt
the .htaccess file in the /data/ folder is empty
Thanks for your thoughts
Last edit: by ironfeather
———–
Publisher of IronFeather Journal since 1987. Host of KGNU Colorado Radio for 20 years.
Currently in Japan & decided to focus on Composr as my number one CMS.
Composr site for community of Hokkaido: Nandalow.com
Composr site for my freelance work: Futurecode.jp
My Compsr edits : http://ironfeather.com/bbs/viewtopic.php?f=12&t=2862
Twitter: https://twitter.com/futurecodejp
Publisher of IronFeather Journal since 1987. Host of KGNU Colorado Radio for 20 years.
Currently in Japan & decided to focus on Composr as my number one CMS.
Composr site for community of Hokkaido: Nandalow.com
Composr site for my freelance work: Futurecode.jp
My Compsr edits : http://ironfeather.com/bbs/viewtopic.php?f=12&t=2862
Twitter: https://twitter.com/futurecodejp
Site director
Our approach is to put out X-Robots-Tag headers, or <meta> declarations, telling search engines if they should not be indexing something. That way people don't need to worry about robots.txt for anything inbuilt to Composr.
In this particular case if a required parameter is missing for notifications.php it doesn't get to the point of putting out that header.
Fixed in:
https://github.com/ocproducts/composr/commit/8f939dd314c684a9f5c5169e08334a94ce492524
Become a fan of Composr on Facebook or add me as a friend. Add me on on Mastodon. Follow me on Minds (where I am most active). Support me on Patreon
- If not, please let us know how we can do better (please try and propose any bigger ideas in such a way that they are fundable and scalable).
- If so, please let others know about Composr whenever you see the opportunity or support me on Patreon.
- If my reply is too Vulcan or expressed too much in business-strategy terms, and not particularly personal, I apologise. As a company & project maintainer, time is very limited to me, so usually when I write a reply I try and make it generic advice to all readers. I'm also naturally a joined-up thinker, so I always express my thoughts in combined business and technical terms. I recognise not everyone likes that, don't let my Vulcan-thinking stop you enjoying Composr on fun personal projects.
- If my response can inspire a community tutorial, that's a great way of giving back to the project as a user.
1 guest and 0 members have just viewed this.
