This site can not be crawled by the Seobility bot. The robots.txt file restricts access for our bot. No URLs can be crawled that deny access to the user agent Seobility or *.
Problem URL: http://shop.nc
## robots.txt for Magento Community and Enterprise ## GENERAL SETTINGS ## Enable robots.txt rules for all crawlers #User-agent: * # Groove Commerce Magento Robots.txt 05/2011 # # robots.txt # # This file is to prevent the crawling and indexing of certain parts # of your site by web crawlers and spiders run by sites like Yahoo! # and Google. By telling these “robots” where not to go on your site, # you save bandwidth and server resources. # # This file will be ignored unless it is at the root of your host: # Used: http://example.com/robots.txt # Ignored: http://example.com/site/robots.txt # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/wc/robots.html # # For syntax checking, see: # http://www.sxw.org.uk/computing/robots/check.html # Website Sitemap Sitemap: https://shop.nc/sitemap.xml # Crawlers Setup User-agent: * Disallow: / User-agent: Googlebot Allow: /sitemap.xml Allow: /sitemap* #Disallow: / # Directories Disallow: /404/ Disallow: /app/ Disallow: /cgi-bin/ Disallow: /downloader/ Disallow: /includes/ #Disallow: /js/ #Disallow: /media/ Allow: /media/catalog/product/ Allow: /media/css/secure/ Allow: /js/ Allow: /js/infortis/jquery/plugins/ Disallow: /lib/ Disallow: /magento/ Disallow: /pkginfo/ Disallow: /report/ #Disallow: /skin/ Disallow: /stats/ Disallow: /var/ Disallow: /blog/ # Paths (clean URLs) Disallow: /index.php/ Disallow: /catalog/product_compare/ Disallow: /catalog/category/view/ Disallow: /catalog/product/view/ Disallow: /catalogsearch/ #Disallow: /checkout/ Disallow: /control/ Disallow: /contacts/ Disallow: /customer/ Disallow: /customize/ Disallow: /newsletter/ Disallow: /poll/ Disallow: /review/ Disallow: /sendfriend/ Disallow: /tag/ Disallow: /wishlist/ # Files Disallow: /cron.php Disallow: /cron.sh Disallow: /error_log Disallow: /install.php Disallow: /LICENSE.html Disallow: /LICENSE.txt Disallow: /LICENSE_AFL.txt Disallow: /STATUS.txt # Paths (no clean URLs) #Disallow: /*.js$ #Disallow: /*.css$ Disallow: /*.php$ #Disallow: /*?*p=*& Disallow: /*?*SID* Disallow: /*?*couleur* Disallow: /*?*marque* Disallow: /*?*limit* Disallow: /*?*dir* Disallow: /*?*mode* Disallow: /*?*sort* Disallow: /*?*marque%5B0%5D=* Disallow: /*?*couleur%5B0%5D=* Disallow: /*?*en_promo%5B0%5D* Disallow: /*?*en_promo* Disallow: /*?*cat* Disallow: /*?*materiaux* Disallow: /*?*price* Disallow: /*?*udropship_vendor* Disallow: /*?*show_add_to_cart* Disallow: /*?*longueur* Disallow: /*?*hauteur* Disallow: /*?*largeur* Disallow: /*?*mode* Disallow: /*?*nomvendeur* Disallow: /*?*matiere_chaussure* Disallow: /*?*pointure* Disallow: /*?*sexe_humain* Disallow: /*?*%5B0%5D=* Disallow: /*?*%5B1%5D=* # Google Image Crawler Setup User-agent: Googlebot-Image Disallow: / Allow: /media/catalog/product/