Show your SEO score to your visitors using the Seobility widget. The widget always displays the current SEO score of your homepage.
# The URL to the DSpace sitemaps # XML sitemap is listed first as it is preferred by most search engines Sitemap: https://repositorio.unal.edu.co/sitemap_index.xml Sitemap: https://repositorio.unal.edu.co/sitemap_index.html ########################## # Default Access Group # (NOTE: blank lines are not allowable in a group record) ########################## User-agent: * # Disable access to Discovery search and filters; admin pages; processes; submission; workspace; workflow & profile page Disallow: /search Disallow: /admin/* Disallow: /processes Disallow: /submit Disallow: /workspaceitems Disallow: /profile Disallow: /workflowitems Disallow: /items/*/request-a-copy # Crawlers should be able to access entity pages, but not the facet search links present on entity pages Disallow: /entities/*?f Disallow: /communities/*?f Disallow: /collections/*?f # Optionally uncomment the following line ONLY if sitemaps are working # and you have verified that your site is being indexed correctly. Disallow: /browse/* # # If you have configured DSpace (Solr-based) Statistics to be publicly # accessible, then you may not want this content to be indexed Disallow: /statistics # # You also may wish to disallow access to the following paths, in order # to stop web spiders from accessing user-based content Disallow: /contact Disallow: /feedback Disallow: /forgot Disallow: /login Disallow: /register ########################## # http://www.bing.com/bingbot.htm User-agent: bingbot Crawl-delay: 5 # https://ahrefs.com/robot/ User-agent: AhrefsBot Crawl-Delay: 10 # https://www.babbar.tech/crawler User-agent: barkrowler Crawl-Delay: 10 # https://help.seranking.com/en/blex-crawler User-agent: BLEXBot Crawl-delay: 10 User-agent: ClaudeBot Crawl-delay: 10 # https://mj12bot.com/ User-agent: MJ12Bot Crawl-Delay: 10 # https://developer.amazon.com/support/amazonbot User-agent: Amazonbot Disallow: / # http://www.apple.com/go/applebot User-agent: Applebot Disallow: / User-agent: dotbot Disallow: / User-agent: meta-externalagent Disallow: / #AwarioBot User-agent: AwarioBot Disallow: / User-agent: AwarioRssBot Disallow: / User-agent: AwarioSmartBot Disallow: / ### ChatGPT https://openai.com/gptbot User-agent: OAI-SearchBot Crawl-delay: 60 User-agent: ChatGPT-User Crawl-delay: 30 User-agent: GPTBot Crawl-delay: 10 ### Sermush http://www.semrush.com/bot.html # To block SemrushBot from crawling your site for different SEO and technical issues: User-agent: SiteAuditBot Disallow: / # To block SemrushBot from crawling your site for Backlink Audit tool: User-agent: SemrushBot-BA Disallow: / # To block SemrushBot from crawling your site for On Page SEO Checker tool and similar tools: User-agent: SemrushBot-SI Disallow: / # To block SemrushBot from checking URLs on your site for SWA tool: User-agent: SemrushBot-SWA Disallow: / # To block SplitSignalBot from crawling your site for SplitSignal tool: User-agent: SplitSignalBot Disallow: / # To block SemrushBot-OCOB from crawling your site for ContentShake AI tool: User-agent: SemrushBot-OCOB Disallow: / # To block SemrushBot-FT from crawling your site for Plagiarism Checker and similar tools: User-agent: SemrushBot-FT Disallow: / ### Perplexity https://docs.perplexity.ai/guides/bots User-agent: PerplexityBot Disallow: / User-agent: Perplexity-User Disallow: / ### Turnitin User-agent: TurnitinBot Disallow: /server/ ### Yandex User-agent: YandexCalendar Disallow: / User-agent: YandexMobileBot Disallow: / User-agent: YandexImages Disallow: / ### Owler User-agent: Owler Disallow: / User-agent: GenAI Disallow: / ############################## # Section for misbehaving bots # The following directives to block specific robots were borrowed from Wikipedia's robots.txt ############################## # advertising-related bots: User-agent: Mediapartners-Google* Disallow: / # Crawlers that are kind enough to obey, but which we'd rather not have # unless they're feeding search engines. User-agent: UbiCrawler Disallow: / User-agent: DOC Disallow: / User-agent: Zao Disallow: / # Some bots are known to be trouble, particularly those designed to copy # entire sites. Please obey robots.txt. User-agent: sitecheck.internetseer.com Disallow: / User-agent: Zealbot Disallow: / User-agent: MSIECrawler Disallow: / User-agent: SiteSnagger Disallow: / User-agent: WebStripper Disallow: / User-agent: WebCopier Disallow: / User-agent: Fetch Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: WebZIP Disallow: / User-agent: linko Disallow: / User-agent: HTTrack Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: Xenu Disallow: / User-agent: larbin Disallow: / User-agent: libwww Disallow: / User-agent: ZyBORG Disallow: / User-agent: Download Ninja Disallow: / # Misbehaving: requests much too fast: User-agent: fast Disallow: / # # If your DSpace is going down because of someone using recursive wget, # you can activate the following rule. # # If your own faculty is bringing down your dspace with recursive wget, # you can advise them to use the --wait option to set the delay between hits. # #User-agent: wget #Disallow: / # # The 'grub' distributed client has been *very* poorly behaved. # User-agent: grub-client Disallow: / # # Doesn't follow robots.txt anyway, but... # User-agent: k2spider Disallow: / # # Hits many times per second, not acceptable # http://www.nameprotect.com/botinfo.html User-agent: NPBot Disallow: / # A capture bot, downloads gazillions of pages with no public benefit # http://www.webreaper.net/ User-agent: WebReaper Disallow: /
Following keywords were found. You can check the keyword optimization of this page for each keyword.
Please select one of the following options for your request.
None of these options suit your need?
You can also download our Quick Start Guide, visit the FAQ section, our Knowledge Base or Blog.
Meta Title
(Nice to have)