Download as
Browse Code »

Commit 4743c28c85f265b8f8e50122dbc9afa1f7bdf48b

Authored by Braulio Bhavamitra
1 parent d08608af
Exists in master and in 28 other branches 1.2+SPB2, 2_split_by_role, add_fields_in_api, api_fixies, api_pagination, api_private_token, api_tasks, article_hit_count, blog_page_bug, checkbox_to_user_can_edit_page, communities_ratings, communities_ratings_moderation, community_ratings_rebase, fix_cloned_article_parent, fix_gallery_image_url, fix_javascript_categories, fix_optional_fields, fix_suggest_article, fix_user_email_validation, highlight_block_fix, notification_plugin, notification_plugin_rebase, script_to_update, split_by_role, stable, temp_ratings, tests, tests_fixies

Block robots on search 

(ActionItem3108)
Showing 1 changed file with 7 additions and 1 deletions   Show diff stats
public/robots.txt
  Diff comments   View file @4743c28
1   -# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
2 1 \ No newline at end of file
  2 +# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  3 +# See also http://en.wikipedia.org/wiki/Robots_exclusion_standard for extensions
  4 +
  5 +User-agent: *
  6 +# block searches' queries
  7 +Disallow: /search*query=*
  8 +Disallow: /cat/*
... ...