Commit 975356ed04970f16abc0b0db6d083e7f749214c3
Exists in
master
and in
22 other branches
Merge branch 'stable' of gitlab.com:noosfero/noosfero into stable
Showing
1 changed file
with
7 additions
and
1 deletions
Show diff stats
public/robots.txt
1 | -# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file | |
2 | 1 | \ No newline at end of file |
2 | +# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file | |
3 | +# See also http://en.wikipedia.org/wiki/Robots_exclusion_standard for extensions | |
4 | + | |
5 | +User-agent: * | |
6 | +# block searches' queries | |
7 | +Disallow: /search*query=* | |
8 | +Disallow: /cat/* | ... | ... |