We have a large public content website, and ChatGPT has crawled without our permissions.<p>How can we block ChatGPT user agent from stealing our written knowledge?
Here's how to make the site accessible to search engines, but not common crawl bots:<p><a href="https://commoncrawl.org/big-picture/frequently-asked-questions/" rel="nofollow">https://commoncrawl.org/big-picture/frequently-asked-questio...</a><p>e.g.:<p><pre><code> User-agent: CCBot
Disallow: /</code></pre>