NoosferoGov / errbit

Authored by Nikita Fedyashev 2014-10-20 03:17:59 +0600

1 parent d50a69c4

Exists in master and in 1 other branch

Showing 1 changed file with 2 additions and 4 deletions Show diff stats

public/robots.txt

1	1	# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
2		-#
3		-# To ban all spiders from the entire site uncomment the next two lines:
4		-# User-Agent: *
5		-# Disallow: /
	2	+User-Agent: *
	3	+Disallow: /
...	...