# AI crawlers — explicit allow # OpenAI / ChatGPT User-agent: GPTBot Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ User-agent: OAI-SearchBot Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ User-agent: ChatGPT-User Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ # Anthropic / Claude User-agent: ClaudeBot Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ # Google AI (Gemini, AIO) User-agent: Google-Extended Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ # Perplexity User-agent: PerplexityBot Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ User-agent: Perplexity-User Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ # ByteDance / TikTok User-agent: Bytespider Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ # Common Crawl (feeds multiple LLM training sets) User-agent: CCBot Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ # Catch-all User-agent: * Allow: / Disallow: /api/ Disallow: /private/ Disallow: /admin/ Host: https://relayfi.com Sitemap: https://relayfi.com/sitemap.xml
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)