The question
When you must decide whether to permit or block bots like GPTBot, ClaudeBot and PerplexityBot.
You are setting robots.txt rules and deciding how to treat AI training and answer-engine crawlers.
Why it matters
Blocking AI crawlers can remove you from AI answers where your buyers now research, while allowing them indiscriminately can expose premium content you monetise elsewhere.
How to decide
IfIf AI answer engines are a growing discovery channel for your audience
ThenYes - allow answer-engine crawlers so you can be cited and recommended.
IfIf you want visibility in AI answers but not model training
ThenYes selectively - allow retrieval bots and block training-only bots per their documented user-agents.
IfIf your core value is paywalled or licence-worthy proprietary content
ThenNo - restrict crawlers on premium paths and expose only marketing content.
IfIf you have no policy and are blocking everything by default
ThenYes - reconsider; a blanket block forfeits AI-search visibility for little gain.