# ============================================== # robots.txt - 仅允许 Google / Bing / DuckDuckGo # ============================================== # 1. 全局规则:禁止所有爬虫 User-agent: * Disallow: / # 2. 放行 Google User-agent: Googlebot Allow: / # 3. 放行 Bing User-agent: bingbot Allow: / # 4. 放行 DuckDuckGo User-agent: DuckDuckBot Allow: / # 5. 屏蔽 AI 训练爬虫(防止内容被大模型抓取) User-agent: GPTBot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: anthropic-ai Disallow: / User-agent: CCBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: PerplexityBot Disallow: / User-agent: Bytespider Disallow: /