Mercurial > rsstweet

view public/robots.txt @ 237:961d362e42c7 legit-client

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
The url in entity media isn't unique as they all point to the same thing
author nanaya <me@nanaya.net>
date Sat, 15 Jul 2023 01:47:34 +0900
parents 1a281427e841
children 89f6c0016bab
line wrap: on
line source

# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
User-agent: *
Disallow: /