宝塔面板Nginx配置拦截AI爬虫蜘蛛机器人，阻止网站内容被AI训练抓取的教程

现在大多数网站80%以上访问量都是AI智能的爬虫蜘蛛机器人，专门提取内容却不给网站带来任何收益的东西，所以我们需要屏蔽AI爬虫蜘蛛机器人。

下面我们以宝塔面板为例，使用Nginx的配置来拦截爬虫蜘蛛机器人，保护原创资源。

宝塔 Nginx 配置（推荐，复制即用）
操作步骤
宝塔 → 网站 → 选你的站点
设置 → 配置文件
在 server { ... } 里、所有 location 块之前 粘贴下面代码
保存 → 重载配置

# ========== 屏蔽 AI / 数据采集爬虫 ==========
set $block_ai_bot 0;

if ($http_user_agent ~* "GPTBot")                          { set $block_ai_bot 1; }
if ($http_user_agent ~* "ChatGPT-User")                     { set $block_ai_bot 1; }
if ($http_user_agent ~* "OAI-SearchBot")                   { set $block_ai_bot 1; }
if ($http_user_agent ~* "anthropic-ai")                     { set $block_ai_bot 1; }
if ($http_user_agent ~* "ClaudeBot")                        { set $block_ai_bot 1; }
if ($http_user_agent ~* "claude-web")                        { set $block_ai_bot 1; }
if ($http_user_agent ~* "Google-Extended")                  { set $block_ai_bot 1; }
if ($http_user_agent ~* "Applebot-Extended")                { set $block_ai_bot 1; }
if ($http_user_agent ~* "Bytespider")                        { set $block_ai_bot 1; }
if ($http_user_agent ~* "CCBot")                            { set $block_ai_bot 1; }
if ($http_user_agent ~* "Amazonbot")                         { set $block_ai_bot 1; }
if ($http_user_agent ~* "FacebookBot")                       { set $block_ai_bot 1; }
if ($http_user_agent ~* "meta-externalagent")                { set $block_ai_bot 1; }
if ($http_user_agent ~* "meta-externalfetcher")              { set $block_ai_bot 1; }
if ($http_user_agent ~* "cohere-ai")                         { set $block_ai_bot 1; }
if ($http_user_agent ~* "Diffbot")                           { set $block_ai_bot 1; }
if ($http_user_agent ~* "ImagesiftBot")                      { set $block_ai_bot 1; }
if ($http_user_agent ~* "Omgilibot")                         { set $block_ai_bot 1; }
if ($http_user_agent ~* "Omgili")                            { set $block_ai_bot 1; }
if ($http_user_agent ~* "PerplexityBot")                     { set $block_ai_bot 1; }
if ($http_user_agent ~* "YouBot")                            { set $block_ai_bot 1; }
if ($http_user_agent ~* "AI2Bot")                            { set $block_ai_bot 1; }
if ($http_user_agent ~* "PetalBot")                           { set $block_ai_bot 1; }
if ($http_user_agent ~* "DataForSeoBot")                      { set $block_ai_bot 1; }
if ($http_user_agent ~* "magpie-crawler")                     { set $block_ai_bot 1; }
if ($http_user_agent ~* "img2dataset")                         { set $block_ai_bot 1; }
if ($http_user_agent ~* "Scrapy")                            { set $block_ai_bot 1; }
if ($http_user_agent ~* "python-requests")                     { set $block_ai_bot 1; }
if ($http_user_agent ~* "curl/")                              { set $block_ai_bot 1; }
if ($http_user_agent ~* "wget")                               { set $block_ai_bot 1; }
if ($http_user_agent ~* "HttpClient")                         { set $block_ai_bot 1; }
if ($http_user_agent ~* "Java/")                              { set $block_ai_bot 1; }
if ($http_user_agent ~* "Go-http-client")                    { set $block_ai_bot 1; }

if ($block_ai_bot = 1) {
    return 403;
}
# ========== 屏蔽结束 ==========

还有个简洁写法，可以使用：

# ========== 屏蔽 AI / 数据采集爬虫 精简版 ==========
if ($http_user_agent ~* "(GPTBot|ChatGPT-User|OAI-SearchBot|anthropic-ai|ClaudeBot|claude-web|Google-Extended|Applebot-Extended|Bytespider|CCBot|Amazonbot|FacebookBot|meta-externalagent|meta-externalfetcher|cohere-ai|Diffbot|ImagesiftBot|Omgilibot|Omgili|PerplexityBot|YouBot|AI2Bot|PetalBot|DataForSeoBot|magpie-crawler|img2dataset|Scrapy|python-requests|curl/|wget|HttpClient|Java/|Go-http-client)") {
    return 403;
}
# ========== 屏蔽结束 ==========

这样就可以屏蔽常见的AI爬虫蜘蛛机器人了，如果有其他的，可以自己继续添加就可以了。