#**************************************************************************** # robots.txt # Purpose: Control how crawlers access the site # Notes: # - Major search engines explicitly allowed # - Trusted SEO tools allowed for audits # - Sensitive Webflow pages blocked # - Spammy / scraper bots disallowed #**************************************************************************** #-------------- # Default Rules #-------------- User-agent: * Disallow: /cgi-bin/ Disallow: /css/ Disallow: /js/ #------------------------------------------ # Major Search Engines — Explicitly Allowed #------------------------------------------ User-agent: Googlebot Allow: / User-agent: Googlebot-Image Allow: / User-agent: Mediapartners-Google Allow: / User-agent: AdsBot-Google Allow: / User-agent: AdsBot-Google-Mobile Allow: / User-agent: Bingbot Allow: / User-agent: msnbot-media Allow: / User-agent: Applebot Allow: / User-agent: Yandex Allow: / User-agent: YandexImages Allow: / User-agent: Slurp Allow: / #---------------------------- # Trusted SEO Tools — Allowed #---------------------------- User-agent: AhrefsBot Allow: / User-agent: AhrefsSiteAudit Allow: / User-agent: SemrushBot Allow: / User-agent: SiteAuditBot Allow: / User-agent: SemrushBot-SA Allow: / User-agent: DotBot Allow: / User-agent: MJ12bot Allow: / User-agent: Screaming Frog SEO Spider Allow: / User-agent: botify Allow: / User-agent: Deepcrawl Allow: / User-agent: lumar Allow: / User-agent: OnCrawl Allow: / User-agent: RyteBot Allow: / User-agent: OnPageBot Allow: / User-agent: Seobility Allow: / User-agent: SeobilityBot Allow: / User-agent: serpstatbot Allow: / User-agent: RSiteAuditor Allow: / User-agent: Sitebulb Allow: / User-agent: Sitebulb Crawler Allow: / #------------------------------ # Blocked Spammy / Scraper Bots #------------------------------ User-agent: PetalBot Disallow: / User-agent: MauiBot Disallow: / User-agent: ZoominfoBot Disallow: / User-agent: GPTBot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: MegaIndex Disallow: / User-agent: Sogou Disallow: / User-agent: Exabot Disallow: / User-agent: CCBot Disallow: / User-agent: IstellaBot Disallow: / User-agent: TurnitinBot Disallow: / #----------------- # Sitemap Location #----------------- Sitemap: https://www.bloomcommercialclean.com/sitemap.xml
Se han encontrado las siguientes palabras clave. Comprueba si esta página está bien optimizada para cada palabra clave en concreto.
(Deseable)