User-agent: * Disallow: # OpenAI (ChatGPT, GPTBot) User-agent: GPTBot Disallow: User-agent: ChatGPT-User Disallow: # Anthropic (Claude) User-agent: ClaudeBot Disallow: # Google AI (Gemini, Bard) User-agent: Google-Extended Disallow: # Perplexity AI User-agent: PerplexityBot Disallow: # xAI / Grok (Twitter/X) — emerging agents User-agent: xAIBot Disallow: User-agent: GrokBot Disallow: User-agent: Twitterbot Disallow: # Common Crawl (used by many LLMs) User-agent: CCBot Disallow: # Amazon (used in AI/ML pipelines) User-agent: Amazonbot Disallow: # Facebook / Meta AI User-agent: facebookexternalhit Disallow: User-agent: Facebot Disallow: # DuckDuckGo AI User-agent: DuckDuckBot Disallow: # Neeva (now part of Snowflake) User-agent: NeevaBot Disallow: # You.com User-agent: YouBot Disallow: # Andi AI User-agent: andi-bot Disallow: # Phind (dev AI search engine) User-agent: PhindBot Disallow: # Bing / Microsoft AI User-agent: bingbot Disallow: User-agent: BingPreview Disallow: # Yandex AI / General crawler User-agent: YandexBot Disallow: # Apple AI systems (Siri, Spotlight) User-agent: Applebot Disallow: # AhrefsBot (used in AI/data SEO) User-agent: AhrefsBot Disallow: Sitemap: https://www.vikingroofingnewjersey.com/sitemap_index.xml
Folgende Keywords wurden erkannt. Überprüfe die Optimierung dieser Keywords für Deine Seite.
(Nice to have)