# Block AI Scrapers User-Agent: AI2Bot User-Agent: Applebot-Extended User-Agent: Bytespider User-Agent: CCBot User-Agent: ClaudeBot User-Agent: Diffbot User-Agent: FacebookBot User-Agent: Google-Extended User-Agent: GPTBot User-Agent: Kangaroo Bot User-Agent: Meta-ExternalAgent User-Agent: omgili User-Agent: PanguBot User-Agent: Timpibot User-Agent: Webzio-Extended User-Agent: Anthropic-ai User-Agent: Claude-Web User-Agent: Cohere-ai Disallow: / User-agent: * # Only allow URLs generated with frontend routing Disallow: /*?id=* Disallow: /*&id=* # Should always be protected, but you know... Disallow: /*/Private/* Disallow: /*/Configuration/* # Disallow all files in /typo3temp/var/ Disallow: /typo3temp/var/* # Disallow all files in /typo3/ Disallow: /typo3/ # Disallow all kind of sql files Disallow: *.sql Disallow: *.sql.gz
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)