Block AI training bots without losing AI answers

Training crawlers and the bots that answer people are usually separate, so you can turn one away and keep the other. For example, this blocks OpenAI's and Anthropic's training crawlers and Common Crawl, while ChatGPT and Claude can still fetch pages to answer people:

User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

Each bot page in the bot directory shows the exact name it looks for in robots.txt.

Check it worked

Open your site's robots.txt page in SenseGap. It reads your live robots.txt and shows, for each bot that actually visits, whether it's allowed and which rule decided. If a blocked bot keeps opening pages, it's ignoring robots.txt, or it's a spoofed bot using that name.