~cytrogen/masto-fe

ref: 1db0b28d178dee1db99ca6fdbc5bccf51f2bf445 masto-fe/public/robots.txt -rw-r--r-- 190 bytes
1db0b28d — Ittihadyya reordered settings in order of \'severity\' \(unfollow, delete, boost, fav\) and added the setting to allow confirmation of favorites 7 months ago 
                                                                                
1
2
3
4
5
6
7
8

# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file

User-agent: GPTBot
Disallow: /

User-agent: *
Disallow: /media_proxy/
Disallow: /interact/