DeveloperSide.NET Forums
May 31, 2020, 06:45:44 PM *
Welcome, Guest. Please login or register.

Login with username, password and session length
   Home   Help Search Login Register  
Pages: [1]
Author Topic: how to unblock google bot?! how to change access rules for server  (Read 28000 times)
0 Members and 1 Guest are viewing this topic.
Posts: 9

View Profile Email
« on: February 10, 2010, 02:58:18 AM »

i get a unreachable robots txt file error in google web master tools. every where i look it comes back to a server config problem relating to bad bots and the rules concerning how many times a bot can acess my site before it thinks it is an attack. I need help with this problem asap.

Thanks a bunch
Master of All Subjects
Posts: 3272

View Profile WWW Email
« Reply #1 on: February 10, 2010, 05:07:00 PM »

You should try to access your-domain\robots.txt URL in your browser to see what you get. It might not even exist. If it does, it can block bots, but only directly, and not via any type of a metric (ex: by # of accesses), so take a look in it.

The only other configuration I can think of that could cause a problem is with mod_security rulesets but it's unlikely.
« Last Edit: February 10, 2010, 05:08:43 PM by admin » Logged

Advanced PHP and MySQL Solutions for your Web Design and Development needs with Web.Developer Server Suite.
Pages: [1]
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.9 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!