Mercurial > rsstweet
diff public/robots.txt @ 40:1a281427e841
Disallow site crawling.
author | edogawaconan <me@myconan.net> |
---|---|
date | Thu, 11 Dec 2014 15:56:16 +0900 |
parents | a1c9fadc0904 |
children | 89f6c0016bab |
line wrap: on
line diff
--- a/public/robots.txt Thu Dec 11 15:51:35 2014 +0900 +++ b/public/robots.txt Thu Dec 11 15:56:16 2014 +0900 @@ -1,5 +1,5 @@ # See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: -# User-agent: * -# Disallow: / +User-agent: * +Disallow: /