Specification of pages to be ignored by web crawlers. See http://www.robotstxt.org/ for details.
Declarations
Type
strings concatenated with "\n"Default
""Example
"Disallow: /foo/"
option
Specification of pages to be ignored by web crawlers. See http://www.robotstxt.org/ for details.
strings concatenated with "\n""""Disallow: /foo/"