Robots.txt
User-agent: *
Crawl-delay: 60
Request-rate: 1/60s
User-agent: *
Disallow: /site/component/search/
Disallow: /component/search/
Disallow: /*?searchword=
Disallow: /*format=opensearch
Disallow: /*task=user
Disallow: /site/component/mailto/
User-agent: Bytespider
Disallow: /
User-agent: meta-externalagent
Disallow: /
User-agent: Meta-ExternalAgent
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: Baiduspider
Disallow: /
User-agent: SemrushBot
Disallow: /
User-agent: YandexBot
Crawl-delay: 10
(Nice to have)