Mercurial > rsstweet

view public/robots.txt @ 203:af84c9f23263

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Fix redirect for twitter user with nil screen name Everything else is still kind of broken but it's kind of beyond help anyway
author nanaya <me@nanaya.pro>
date Mon, 19 Oct 2020 03:42:22 +0900
parents 1a281427e841
children 89f6c0016bab
line wrap: on
line source

# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
User-agent: *
Disallow: /