Commit 6d6dc80da258b039a71543f4469b0a9cc4216912
1 parent
d50a69c4
Exists in
master
and in
1 other branch
By all crawlers by default
Showing
1 changed file
with
2 additions
and
4 deletions
Show diff stats
public/robots.txt
| 1 | # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file | 1 | # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file |
| 2 | -# | ||
| 3 | -# To ban all spiders from the entire site uncomment the next two lines: | ||
| 4 | -# User-Agent: * | ||
| 5 | -# Disallow: / | 2 | +User-Agent: * |
| 3 | +Disallow: / |