# AI assistant / LLM crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, # Claude-User, PerplexityBot, Google-Extended, Applebot-Extended, # Meta-ExternalAgent, Bytespider, CCBot, Amazonbot, DuckAssistBot, etc.) are # welcome — everything not disallowed below is crawlable. # Deliberately no per-bot groups: a named "User-agent" group replaces the "*" # group for that bot, which would exempt it from the junk-path disallows and # waste its crawl budget on tracking endpoints. # Machine-readable entry points for LLMs and agents: # /llms.txt (index), /llms-full.txt (full text of cornerstone guides), # append .md to any article URL for its raw Markdown. User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=yes Disallow: /*/actions_panel* Disallow: /users/auth/twitter* Disallow: /users/auth/github* Disallow: /report-abuse Disallow: /connect/@* Disallow: /search?q=* Disallow: /search/?q=* Disallow: /search/feed_content Disallow: /listings*?q=* Disallow: /mod/* Disallow: /mod?* Disallow: /admin/* Disallow: /reactions Disallow: /async_info/ Disallow: /ahoy/ Disallow: /auth_pass/* Disallow: /fallback_activity_recorder Disallow: /page_views Disallow: /null Sitemap: https://www.promptzone.com/sitemap-index.xml