Librotea.eldiario.es - SEO Check

Meta Tags

Medienliste

Überschriftenstruktur

Gefundene Links auf dieser Seite

HTTP-Header

Links von Wikipedia

• Silvia Sesé

Robots.txt

# modified at 21-02-2025
# Name: robots.txt
#
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
#
# This file will be ignored unless it is at the root of your host:
# Used:    http://example.com/robots.txt
# Ignored: http://example.com/site/robots.txt
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/wc/robots.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html

User-agent: *
Allow: /
Allow: /wp-sitemap.xml
Allow: /estanterias/*
Allow: /inspiradores/*
Allow: /articulos/*
Allow: /libros/*
Allow: /*.php$
Allow: /*.js$
Allow: /*.inc$
Allow: /*.css$
Allow: /*.xhtml$
Allow: /*.gif$
Allow: /*.png$
Allow: /*.jpeg$
Allow: /*.jpg$
Allow: /*.ico$
### No incluir la página de búsqueda.
Disallow: /buscar
### No incluir la página de restablecer contraseña.
Disallow: /restablecer-contrasena
### Excluir rutas de wordpress
Disallow: /wp-content/

### No carpetas de recursos estáticos.
Disallow: /bundles/
Disallow: /css/
Disallow: /fonts/
Disallow: /sites/
Disallow: /images/
Disallow: /js/

### Site maps
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-articles-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-bks-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-bnw-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-edt-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-ent-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-google-news-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-images-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-ntv-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-nws-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-shv-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-tags-index.xml
Sitemap: https://librotea.eldiario.es/sitemap/esp-sitemap-videos-index.xml

## Block admin section
Disallow: /admin/

### Otros bots
User-agent: Genio
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: ScooperBot
Disallow: /

User-agent: rogerbot
Disallow: /

User-agent: Flamingo_SearchEngine
Disallow: /

User-agent: Facebot
Disallow: /

User-agent: LuminateBot
Disallow: /

User-agent: Vagabondo
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: SeznamBot
Disallow: /

User-agent: R6_CommentReader
Disallow: /

User-agent: Yeti
Disallow: /

User-agent: heritrix
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: ShowyouBot
Disallow: /

User-agent: GozaikBot
Disallow: /

User-agent: python-requests
Disallow: /

User-agent: QuerySeekerSpider
Disallow: /

User-agent: DotBot
Disallow: /

User-agent: YandexImages
Disallow: /

User-agent: Apache-HttpClient
Disallow: /

User-agent: PiplBot
Disallow: /

User-agent: Scrapy
Disallow: /

User-agent: Buck
Disallow: /

User-agent: WikiDo
Disallow: /

User-agent: ZoominfoBot
Disallow: /

User-agent: Sogou
Disallow: /

User-agent: Zend_Http_Client
Disallow: /

User-agent: robots
Disallow: /

User-agent: arquivo-web-crawler
Disallow: /

User-agent: bidswitchbot
Disallow: /

User-agent: G-i-g-a-b-o-t
Disallow: /

User-agent: Gigabot
Disallow: /

User-agent: GarlikCrawler
Disallow: /

User-agent: CAAM
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: Clickagy Intelligence Bot
Disallow: /

User-agent: Jersey
Disallow: /

User-agent: libwww-perl
Disallow: /

User-agent: ltx71
Disallow: /

User-agent: omgili
Disallow: /

User-agent: PiplBot
Disallow: /

User-agent: Python-urllib
Disallow: /

User-agent: ZoominfoBot
Disallow: /

Suchvorschau