Mercurial > rsstweet

comparison public/robots.txt @ 254:89f6c0016bab default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Update to latest rails
author nanaya <me@nanaya.net>
date Tue, 03 Sep 2024 19:21:25 +0900
parents 1a281427e841
children
comparison
equal deleted inserted replaced
253:d726e8b92dd1 254:89f6c0016bab
1 # See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file 1 # See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
2 #
3 # To ban all spiders from the entire site uncomment the next two lines:
4 User-agent: * 2 User-agent: *
5 Disallow: / 3 Disallow: /