Shopify robots.txt for AI Crawlers: Complete Configuration Guide (2026)

Why robots.txt Matters for AI Commerce

Your robots.txt file sits at yourdomain.com/robots.txt and tells web crawlers which parts of your site they can access. For AI commerce, it now includes a dozen AI-specific crawlers that determine whether your products appear in ChatGPT, Perplexity, Gemini, and other AI platforms.

If your robots.txt blocks these crawlers, your products are invisible to AI — even if Agentic Storefronts is enabled and your product data is perfect.

Shopify manages robots.txt for you and allows major AI crawlers by default. But third-party apps, custom theme modifications, and robots.txt Liquid templates can override these defaults.

The Three Tiers of AI Crawlers

Tier 1 — Commerce-critical. OAI-SearchBot (ChatGPT Shopping), Googlebot (Google AI Mode), Bingbot (Microsoft Copilot). Blocking these means your products won’t appear in their AI shopping channels.

Tier 2 — Discovery and citation. GPTBot (general ChatGPT), PerplexityBot, ClaudeBot, ChatGPT-User. Allowing these expands visibility beyond shopping into general AI answers.

Tier 3 — Emerging. Applebot-Extended (Apple Intelligence), Meta-ExternalAgent (Meta AI), Bytespider (TikTok AI), YouBot (You.com). Future channels worth keeping open.

How to Check Your Current robots.txt

Visit yourdomain.com/robots.txt in your browser. Look for any Disallow rules targeting AI crawler user-agents.

If you see “User-agent: GPTBot” followed by “Disallow: /” — that crawler is completely blocked. If there’s no mention of a specific AI crawler, Shopify’s default configuration applies, which generally allows them.

Common problems: blanket Disallow rules from privacy-focused apps, custom robots.txt Liquid templates overriding Shopify defaults, and outdated configurations.

Geopad’s free audit tool checks your robots.txt automatically as part of its GEO readiness score.

How to Fix Your robots.txt on Shopify

Shopify gives you control through the robots.txt.liquid template in your theme. Go to Online Store, then Themes, then Actions, then Edit Code, and look for the robots.txt.liquid file.

If the file doesn’t exist, Shopify uses its default configuration, which allows all major crawlers. If it does exist, review it carefully for any Disallow rules targeting AI crawlers.

The safest approach is to explicitly allow all AI crawlers. Add User-agent lines for each crawler you want to permit, followed by “Allow: /”.

Important: robots.txt changes take effect immediately, but AI crawlers may not re-visit for days or weeks.

robots.txt vs Agentic Storefronts

Agentic Storefronts syndicates your product data through Shopify Catalog via structured data feeds. robots.txt controls whether AI crawlers can access your actual website pages for supplementary signals.

Think of Agentic Storefronts as the product listing and robots.txt as permission to visit your store for additional context. You want both working.

llms.txt: The robots.txt Companion for AI

While robots.txt tells AI what it can access, llms.txt tells AI what it should understand. The llms.txt standard provides a structured overview of your business specifically for AI consumption.

Geopad generates and hosts your llms.txt automatically via Shopify App Proxy, serving it from your own domain. No separate hosting needed.