Mercurial > rsstweet

view public/robots.txt @ 66:1161ad2a4390

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Take care of rt removal from gem api Double the default count since filtering is done at gem level, not api.
author nanaya <me@myconan.net>
date Tue, 01 Dec 2015 01:34:12 +0900
parents 1a281427e841
children
line wrap: on
line source

# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
User-agent: *
Disallow: /