Robots text question

Webmaster Discussion
, ,
#1

When you ‘disallow’ a folder in your robots . txt file, does that include all sub folders or do you need to add a disallow for them as well?:help:

#2

Re: Robots text question

You asked the same exact question less than a month ago…

http://www.gaydemon.biz/showthread.php?19171-Robots-txt-question

:whistle:

#3

Re: Robots text question

[QUOTE=rawTOP;101701]You asked the same exact question less than a month ago…

http://www.gaydemon.biz/showthread.php?19171-Robots-txt-question

:whistle:[/QUOTE]

I plead Old Age setting in :blushing:

#4

Re: Robots text question

There are a few tools you can use to help you create them if you’re not sure yourself, for example:

[URL=“http://tools.seobook.com/robots-txt/generator/”]http://tools.seobook.com/robots-txt/generator/
[URL=“http://www.mcanerin.com/en/search-engine/robots-txt.asp”]http://www.mcanerin.com/en/search-engine/robots-txt.asp

There is also a Robot Generator in Google Webmaster Tools:

http://www.google.com/webmasters/tools/

And a good article by them about Robots, somewhat old but still valid:

http://googlewebmastercentral.blogspot.com/2008/03/speaking-language-of-robots.html