How to Track AI Traffic from ChatGPT, Perplexity, and Claude

AI traffic is two separate measurements: AI crawlers reading your pages, and AI assistants sending you visitors. Track crawlers by user agent in server-side logs, and referrals through GA4's AI Assistants channel or referrer values. On two sites we run, the crawlers and the referrers turned out to be almost entirely different systems.

That last sentence is the part nobody warned me about, so let me start there rather than with a definition.

What you'll need: access to your analytics tool, permission to edit robots.txt, and either server-side request logs or an analytics tool that records bot requests at the edge. Budget 30 to 60 minutes for the referral setup and another hour if you also want crawler reporting. Intermediate — you should be comfortable in GA4 admin.

Here's the short version of the process, in case you want the outline before the detail:

1. Decide which question you're answering — crawlers and referrals need different plumbing
2. Track AI referrals — GA4's AI Assistants channel, or referrer values in any analytics tool
3. Track AI crawlers — by user agent, from logs your tracking script can't see
4. Set up the GA4 channel — and understand what it deliberately leaves out
5. Account for the traffic that lands as direct — your AI number is a floor, not a total
6. Connect AI traffic to value — sessions alone won't justify anything
7. Decide what to block — after you know who's actually visiting

AI traffic is two different things, and most trackers measure one

When someone says "we got 400 visits from ChatGPT last month," they mean AI referrals: a person read an answer, clicked a link, and landed on the site. That's a human, and your analytics tool sees them like any other visitor.

AI crawlers are the other half. These are automated agents fetching your pages — to train a model, to build a search index, or to answer one specific question a user just asked. No human arrives. Nothing shows up in your sessions report.

These get conflated constantly, and the confusion has a cost: most advice about "controlling AI traffic" is about crawlers, while most reporting about "AI traffic value" is about referrals. They're different systems with different fixes.

They're also measured in different places. Referrals arrive through your tracking script like any visitor. Crawlers mostly don't run JavaScript, so a script-only analytics tool can't see them at all — you need server-side or edge-level request logging, which is why bot detection has to sit below the browser. If your tool only reports what the browser executed, its AI crawler number isn't low. It's absent.

What we measured: the crawler reading you isn't the assistant referring you

I pulled live data from two sites my team runs — not this one. Both are English-language B2B content sites: one covers SEO tooling, the other technology data. I'm keeping the domains out of this, but the numbers are as measured on 4 August 2026. Both windows begin in July 2026 and run to 3 August; they're different lengths, so I've reported the two sites separately rather than blending them.

Two sites over a few weeks is a measurement, not a benchmark. Your mix will differ. But the shape of the result was consistent across both, and it wasn't what I expected.

AI referrals — which assistants actually sent visitors:

AI crawlers — which agents actually fetched pages:

Site B also logged 4 requests from Cursor and 2 from Meta-ExternalFetcher, which I've left out of the table for readability but kept in the totals below.

Read those two tables against each other. ChatGPT was the largest single source of AI-referred visits on both sites — 373 sessions between them — and GPTBot made zero requests to either. Claude sent 118 visits off nine ClaudeBot requests.

Meanwhile the three agents doing almost all the actual crawling — Amazonbot, Bytespider, and Meta-ExternalAgent — accounted for 6,425 of the 6,507 named AI-crawler requests across both sites, or 98.7%. Between them they sent no referred visits at all.

Why would ChatGPT cite pages its training crawler never fetched? Because GPTBot isn't the agent that does that job. OpenAI runs four separate agents: GPTBot trains foundation models, OAI-SearchBot surfaces pages in ChatGPT's search, ChatGPT-User fetches a page when a person asks, and OAI-AdsBot checks submitted ad pages. Anthropic splits the same way — ClaudeBot for training, Claude-User for user-directed fetches, Claude-SearchBot for search quality.

So a platform can send you traffic while its training crawler ignores you completely. That single fact undoes most of the "block GPTBot to control AI traffic" advice in circulation.

One more thing worth noticing: AI crawlers were a minority of bot traffic on both sites — 12.6% of 41,107 bot requests on Site A, 5.3% of 25,525 on Site B. Headless browsers were the biggest bot category on both, at 49% and 80%. If you're worried about bots eating your bandwidth, AI agents may not be your problem.

Step 1: Decide which question you're actually answering

Pick one before you touch a setting. The two questions need different data sources, and building the wrong one is the most common way this goes wrong.

Which report answers which question

"Is AI sending me visitors?" is a referral question. It's answered in your analytics tool, from referrer headers and UTM parameters, and it's about humans and conversions.

"Are AI systems reading my content?" is a crawler question. It's answered from server-side or edge request logs, by user agent, and it's about consent, bandwidth, and whether you're eligible to appear in AI answers at all.

Most teams want both, but they want them for different reasons and on different schedules. Referral tracking is a weekly marketing report. Crawler tracking is a quarterly policy review.

You'll know you picked right when you can say what you'd do differently based on the number. If a crawler count wouldn't change your robots.txt, you wanted the referral report.

Where teams get this wrong

• Building crawler reporting inside a JavaScript-only tool. It will report near-zero and you'll conclude no AI system reads your site. It just can't see them.
• Treating crawler volume as an audience metric. A crawl is not a reader, and it's not a citation either.

Step 2: Track AI referrals from ChatGPT, Perplexity, Claude, and Gemini

Referral tracking works because assistants pass identifiable referrer values, and some append their own campaign parameters.

Referrer values for each assistant

Here's what to match on:

If your analytics tool has a built-in AI report, use it — you'll get the split without maintaining a regex. In tinyanalytics this is an AI referrals breakdown alongside your other channels, and it's what produced the first table in this post. Attribution is applied when the report is read rather than stored as a tag at collection time, so turning it on shows you your history rather than starting a new count from today. Bot requests stay in a separate report, so crawlers never inflate your session numbers.

If you're maintaining your own matcher, keep it to hostnames rather than trying to catch every possible AI product. The long regexes floating around, with fifteen alternations covering tools nobody uses, mostly add maintenance cost and false positives.

You'll know it's working when a test click from a real ChatGPT answer shows up in your report within a few minutes, attributed to ChatGPT rather than direct.

Why some ChatGPT clicks never look like ChatGPT

• Matching openai.com broadly. That catches OpenAI's marketing site, not just assistant referrals. Match chatgpt.com and chat.openai.com.
• Forgetting the in-app browsers. A ChatGPT click on iOS often opens in an in-app webview, and referrer behaviour there is inconsistent. Some of those visits will land as direct no matter what you configure.

One thing that does help: tag the links you control. If you publish comparison pages or documentation you expect to get cited, adding your own UTM to canonical URLs won't help — assistants strip and rewrite those. What does help is making sure each page has a distinct, stable URL, because that's the string that shows up in your landing-page report when the referrer is missing.

Step 3: Track AI crawlers by user agent

Crawler tracking happens below the browser. You need request logs — from your server, CDN, or an analytics tool that records bot requests at the edge rather than from JavaScript.

The user agents to match

Match on these user agent tokens:

The distinction inside each operator's set is the part that matters. Perplexity documents that PerplexityBot exists to surface and link sites, explicitly not to train models — and that Perplexity-User "generally ignores robots.txt rules" because a person initiated the fetch. OpenAI says much the same about ChatGPT-User.

Verifying a crawler is who it claims to be

User agent strings are trivially forged, so every operator publishes IP ranges for verification: OpenAI at openai.com/gptbot.json and its siblings, Anthropic at claude.com/crawling/bots.json, Perplexity at perplexity.com/perplexitybot.json. If a crawler number looks implausible, check the IPs before acting on it.

You'll know it's working when your crawler report lists agents you didn't expect. Mine listed Amazonbot at the top, which I'd never have guessed. Two traps here: assuming the big three AI names dominate, when on both sites I measured they didn't come close, and counting user agents without verifying IPs, because scrapers impersonate these bots constantly.

Step 4: Set up the AI Assistants channel in GA4

GA4 now ships this natively, which makes most of the custom-regex tutorials online redundant. Google's default channel group documentation defines an AI Assistants channel: traffic arriving "from sources like ChatGPT, Gemini, Deepseek, Copilot, or Grok." When GA4 recognises the referrer it sets medium to ai-assistant and campaign to (ai-assistant).

Finding the channel in GA4

Open GA4, go to Reports → Acquisition → Traffic acquisition, and set the primary dimension to Session default channel group. AI Assistants appears as its own row once you have qualifying sessions.

You'll know it's working when that row appears and its session count roughly matches what you see filtering referrals by chatgpt.com manually.

If you already built a custom channel group from an older tutorial, remove it. It can shadow the built-in channel and split your data across two rows. And give it time — GA4 processing lag applies here as everywhere, so see why GA4 data arrives late if the row stays empty longer than you expect.

What the AI Assistants channel leaves out

Here's the part Google states plainly and almost no guide repeats: the channel excludes Google's AI Overviews and AI Mode. Clicks from Google's own AI surfaces are not in this number. They land in organic search.

That exclusion changes what the number means. "AI Assistants" in GA4 is third-party assistant traffic, not all AI-influenced traffic. If you report it as "our AI traffic" to someone who assumes AI Overviews are included, you're both misreading it.

Step 5: Account for the AI traffic that lands as direct

Whatever number you end up with is a floor. A meaningful share of assistant click-throughs arrive with no referrer at all, and those land in direct.

Why referrers go missing

Three mechanisms cause it. Assistants running inside native mobile apps open links in webviews that often don't pass a referrer. Some assistants route clicks through a redirect that drops the header. And any navigation from a page that sets a restrictive referrer policy loses it too.

You can't recover those sessions precisely. What you can do is watch direct traffic for the tell-tale pattern: a jump in direct sessions landing on deep content pages rather than your homepage, with engagement patterns matching your known AI referrals. Deep-page direct traffic is unusual for most sites, and it's the clearest signal you'll get.

How to report a number you can't fully see

Be careful with the wording. "At least 370 sessions came from AI assistants" is honest. "AI drives 3% of our traffic" implies a precision the data doesn't support in either direction — some AI traffic is hidden in direct, and some direct traffic has nothing to do with AI.

Two ways to get this wrong: attributing all deep-page direct traffic to AI, when email clients, PDFs, and chat apps strip referrers too; and adding an estimated correction factor. I've seen people multiply their AI number by two to "account for direct." That's inventing data.

Step 6: Connect AI traffic to what it's worth

Session counts don't justify any decision. What matters is whether those visits do anything.

Segmenting AI traffic by outcome

Set up conversion tracking that segments by channel, then compare AI-referred sessions against your other channels on the outcomes you actually care about: signups, trials, purchases, documentation reads that precede activation. In tinyanalytics you'd do this with a goal or funnel filtered to the AI channel; in GA4 it's a comparison on the AI Assistants channel.

I'd resist quoting anyone else's conversion multiple for AI traffic, including the ones circulating in vendor marketing. The number depends entirely on what you sell and who asks assistants about it. Measure yours.

When your sample is too small to trust

At 333 and 370 sessions across several weeks, the two sites I pulled don't have enough AI-referred conversions to say anything reliable about conversion rate. That's the honest finding, and I'd rather report it than dress it up. If your AI traffic is at that scale, hold off on conclusions for another quarter.

Two related traps: optimising hard for a channel that's 1-3% of sessions, and comparing AI conversion rate to your site average on a handful of conversions, where a couple of signups swing the rate wildly.

Step 7: Decide what to block, now that you know who's visiting

Do this last, because the measurement changes the decision.

A robots.txt that blocks training but keeps citation

robots.txt gives you per-agent control, and the agents do different jobs:

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: ClaudeBot
Disallow: /

User-agent: PerplexityBot
Allow: /

That configuration opts out of training while staying eligible for citation in ChatGPT search and Perplexity results. OpenAI notes robots.txt changes can take around 24 hours to take effect.

The two limits of robots.txt

User-initiated agents — ChatGPT-User, Claude-User, Perplexity-User — may fetch regardless, and Perplexity documents this outright. And robots.txt is a request, not enforcement; crawlers that ignore it need blocking at the CDN or firewall.

The measurement matters here because the agents most likely to be consuming your bandwidth may not be the ones in the tutorials. On the sites I checked, blocking GPTBot and PerplexityBot would have changed nothing — they weren't visiting. Blocking Amazonbot and Bytespider would have removed about two-thirds of AI crawler load. Pull your own crawler report before writing rules.

What AI traffic data can't tell you

Three limits, and they're not small.

A crawl is not a citation. An AI crawler report shows a crawler read a page, not that the page was cited in any answer. Crawl volume and citation frequency aren't the same measurement, and nothing in your logs bridges them.

A referral is not the full picture. AI traffic measures click-throughs from assistant answers, not unseen citations. If an assistant summarises your page and the user never clicks, that influence exists and is invisible to every analytics tool, including ours.

You can't see the prompt. Assistants don't pass the question that produced the answer. Tools claiming to show you "the prompts driving your traffic" are inferring, not observing.

There's also the ordinary measurement caveat: cookieless visitor counts are useful estimates, not exact counts. People on the same network and browser can merge; one person changing networks or browsers can split. That applies to AI-referred visitors like any other.

Tools mentioned in this guide

Frequently asked questions

How do I know if ChatGPT is sending me traffic? Check your analytics tool for referrals from chatgpt.com or the UTM parameter utm_source=chatgpt.com, which ChatGPT appends to many cited links. In GA4, look for the AI Assistants row under Session default channel group. If you see zero and you publish content assistants would plausibly cite, check whether the sessions are landing in direct instead.

What is GPTBot, and should I block it? GPTBot is OpenAI's training crawler. Blocking it signals that your content shouldn't be used to train models — and it does not remove you from ChatGPT's search results, which is OAI-SearchBot's job. That distinction is the whole decision. Block GPTBot if you object to training use; keep OAI-SearchBot allowed if you want to remain citable.

Why does my AI traffic show as direct? Referrer headers get stripped in several situations: assistants running in native mobile apps, redirect hops that drop the header, and restrictive referrer policies. A share of AI click-throughs will always land as direct. Watch for direct traffic arriving on deep content pages — that's the closest signal you'll get.

What's the difference between an AI crawler and an AI referral? A crawler is an automated agent fetching your page with no human involved. A referral is a person who read an AI answer and clicked through. Crawlers appear in server logs; referrals appear in your analytics sessions. They're separate measurements, and as our own data shows, the platforms doing each can be almost entirely different.

Can I see which prompt someone used to find my site? No. Assistants don't pass the user's question in the referrer or any other header. Any tool presenting "the prompts driving your traffic" is modelling likely queries, not reading actual ones.

Does blocking AI crawlers stop AI traffic? Not reliably. User-initiated agents may fetch regardless of robots.txt — Perplexity documents this for Perplexity-User, and OpenAI says the same about ChatGPT-User. And robots.txt is a request rather than enforcement, so agents that ignore it need blocking at the CDN or firewall.

Where to start

Pull your crawler report first, before you change any rule. On the two sites I measured, the crawlers everyone writes about weren't visiting, and the ones consuming real bandwidth never get mentioned. Your list will differ from mine — that's the point of looking.

Then set up referral tracking, give it a quarter, and hold off on conclusions until the sample is big enough to carry them.

tinyanalytics reports both halves in one place: AI referrals beside your other channels, AI crawlers in a separate bot report so they never inflate your session counts. No cookies, no raw IP storage, no cross-site profile — you can see how we handle data in our trust center. See AI traffic or, if you're weighing this against your current setup, compare the alternatives to Google Analytics.