# # Production robots.txt file www.queensland.com # Consumer site. # # Major Search Engines and Known Friendly Spiders User-agent: Googlebot User-agent: googlebot-image User-agent: googlebot-mobile User-agent: MSNBot User-agent: psbot User-agent: Slurp User-agent: yahoo-mmcrawler User-agent: yahoo-blogs User-agent: baiduspider User-agent: Baiduspider-image User-agent: Yandex User-agent: Teoma User-agent: Twiceler User-agent: Gigabot User-agent: Scrubby User-agent: Robozilla User-agent: Bingbot Disallow: /bin/ # Known Unwanted Spiders User-agent: Ahrefs Disallow:/ User-agent: AhrefsBot Disallow:/ User-agent: AlphaBot Disallow:/ User-agent: Alpha search agent Disallow:/ User-agent: Blexbot Disallow:/ User-agent: HubSpot Connect Disallow:/ User-agent: HubSpot Crawler Disallow:/ User-agent: HubSpot Links Crawler Disallow:/ User-agent: HubSpot Marketing Grader Disallow:/ User-agent: HubSpot Webcrawler Disallow:/ User-agent: HubSpot Website Grader Disallow:/ User-agent: Ltx71 Disallow:/ User-agent: MauiBot Disallow:/ User-agent: MegaIndex.ru Disallow: / User-agent: MegaIndex.com Disallow: / User-agent: MJ12bot Disallow:/ User-agent: Semrush Disallow:/ User-agent: SemrushBot Disallow:/ User-agent: Turnitin Disallow:/ User-agent: TurnitinBot Disallow:/ User-agent: Vagabondo Disallow:/ User-agent: VelenPublicWebCrawler Disallow: / User-agent: ZoomInformation Bot Disallow:/ #all others. same same, only crawl-delay User-agent: * Disallow: /bin/ Crawl-delay: 5 #sitemap entry Sitemap: https://www.queensland.com/sitemap.xml Host: www.queensland.com Sitemap: https://www.queensland.com/au-en-aem-sitemap.xml Sitemap: https://www.queensland.com/au-en-atdw-sitemap.xml
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)