# mycsharp.de - Robots Exclusion Standard # https://www.robotstxt.org/ # 2026-06-20T10:33:40.9606558+00:00 # =================================== # AI Chat Bots (User Access) - ALLOW # =================================== User-agent: ChatGPT-User Allow: / User-agent: Claude-Web Allow: / # =================================== # AI Training Crawlers - BLOCK # =================================== User-agent: GPTBot Disallow: / User-agent: ChatGPT Disallow: / User-agent: anthropic-ai Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: CCBot Disallow: / User-agent: cohere-ai Disallow: / User-agent: cohere-training-data-crawler Disallow: / User-agent: FacebookBot Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: Bytespider Disallow: / User-agent: Applebot-Extended Disallow: / User-agent: Amazonbot Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Diffbot Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / # =================================== # SEO & Marketing Bots (optional blocks) # =================================== User-agent: AhrefsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: Mediatoolkitbot Disallow: / User-agent: magpie-crawler Disallow: / User-agent: AwarioBot Disallow: / User-agent: DataForSeoBot Disallow: / # =================================== # Good Bots - Allow with Restrictions # =================================== User-agent: Googlebot User-agent: Googlebot-Image User-agent: Googlebot-Mobile User-agent: Bingbot User-agent: Slurp User-agent: DuckDuckBot User-agent: Baiduspider User-agent: YandexBot # Private/sensitive areas Disallow: /konto Disallow: /account Disallow: /login Disallow: /messages/ Disallow: /forum/posts/*/melden Disallow: /forum/posts/*/report Disallow: /users/*/melden Disallow: /users/*/report Disallow: /forum/threads/*/antworten Disallow: /forum/threads/*/reply Disallow: /forum/threads/*/notifications/ Disallow: /impressum Disallow: /datenschutz Disallow: /cookies Disallow: /api/ Disallow: /ws/ Disallow: /stream/ Disallow: /admin/ Disallow: /internal/ Disallow: /console/ Disallow: /dashboard/ Allow: / # =================================== # All Other Bots - Default Rules # =================================== User-agent: * # Private/sensitive areas Disallow: /konto Disallow: /account Disallow: /login Disallow: /messages/ Disallow: /forum/posts/*/melden Disallow: /forum/posts/*/report Disallow: /users/*/melden Disallow: /users/*/report Disallow: /forum/threads/*/antworten Disallow: /forum/threads/*/reply Disallow: /forum/threads/*/notifications/ Disallow: /impressum Disallow: /datenschutz Disallow: /cookies Disallow: /api/ Disallow: /ws/ Disallow: /stream/ Disallow: /admin/ Disallow: /internal/ Disallow: /console/ Disallow: /dashboard/ # Crawl delay for other bots Crawl-delay: 10 Allow: / # =================================== # Sitemaps # =================================== Sitemap: https://mycsharp.de/sitemap.xml
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)