OpenAI and Anthropic are ignoring robots.txt

18 pointsby Handy-Man11 months ago

3 comments

joshstrange11 months ago

<a href="http://archive.today/bVgFO" rel="nofollow">http://archive.today/bVgFO</a>

arthurcolle11 months ago

robots.txt is a suggestion not a rule

评论 #40758759 未加载

Handy-Man11 months ago

Title editorialized due to being too long