# https://www.robotstxt.org/robotstxt.html User-agent: CCBot Disallow: / User-agent: img2dataset Disallow: / User-agent: Google-Extended Disallow: / User-agent: anthropic-ai Disallow: / User-agent: Claude-Web Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: FacebookBot Disallow: / User-agent: Bytespider Disallow: / User-agent: magpie-crawler Disallow: / # General rules for all other bots User-agent: * # Place allows first to avoid bots skipping after Disallow: / # Allow exactly the homepage Allow: /$ # Allow the homepage with any query parameters Allow: /?* Allow: /api/share/og/* Allow: /g/* Allow: /s/* Allow: /share/* Allow: /canvas/shared/* Allow: /images/* Allow: /auth/* Allow: /gpts$ Allow: /codex$ Allow: /search$ Allow: /backend-anon/* Allow: /public-api/* Allow: /sitemap.xml Allow: /students Allow: /api/public_content/* Allow: /backend-api/public_content/* Allow: /?ref=dotcom # Now block everything else Disallow: / # Specific disallows (redundant for some bots, but still useful for those that respect precedence) Disallow: /auth/logout Disallow: /auth/login?* Disallow: /backend-anon/sentinel/* Disallow: /backend-anon/conversation$ Disallow: /account-link/* Sitemap: https://chatgpt.com/sitemap.xml
Following keywords were found. You can check the keyword optimization of this page for each keyword.
(Nice to have)