DeveloperSide.NET Forums

DeveloperSide.NET => Web.Developer Server Suite Community Edition,
Public Support Forum => Topic started by: colubragens on February 10, 2010, 02:58:18 AM



Title: how to unblock google bot?! how to change access rules for server
Post by: colubragens on February 10, 2010, 02:58:18 AM
i get a unreachable robots txt file error in google web master tools. every where i look it comes back to a server config problem relating to bad bots and the rules concerning how many times a bot can acess my site before it thinks it is an attack. I need help with this problem asap.

Thanks a bunch


Title: Re: how to unblock google bot?! how to change access rules for server
Post by: admin on February 10, 2010, 05:07:00 PM
You should try to access your-domain\robots.txt URL in your browser to see what you get. It might not even exist. If it does, it can block bots, but only directly, and not via any type of a metric (ex: by # of accesses), so take a look in it.

The only other configuration I can think of that could cause a problem is with mod_security rulesets but it's unlikely.