Bot and AI Crawler Analytics

The Bots & AI crawlers report shows detected automated traffic: how many bot requests your site gets, what share of recorded traffic they represent, and which bots and AI crawlers visit. With bot blocking enabled, detected bots stay out of your human sessions, visitor counts, and live counter. Detection is best effort, so bots that evade it can still affect those numbers.

​

What do the overview cards show?

The report opens with eight cards summarizing automated traffic for the current window and filters. The first two are totals; the remaining six count how many requests each detection layer caught.

​

How does TinyAnalytics detect bots?

TinyAnalytics runs six detection layers over every request, listed in the table above. They range from the obvious — a crawler that announces itself in its user agent — to signals that catch bots actively hiding, like datacenter networks, missing browser headers, crawl-shaped request bursts, and automation flags in the browser runtime. A request flagged by any layer appears in bot activity. Because the layers overlap, one request can be caught by several at once, so the six layer counts can add up to more than the Bot requests total. With bot blocking enabled, detected bots are kept out of your human session counts and live visitor counter. Detection is best effort: bots that evade the available signals can still affect your reports.

​

Which AI crawlers are reading my site?

The report leads with an AI crawlers section naming the AI bots that visited — GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, PerplexityBot, Google-Extended, Bytespider, CCBot, and others. A crawler that hides its user-agent but comes from a known AI network is still counted and named by its operator (such as Anthropic or OpenAI). AI crawlers that can’t be named individually roll up into Other AI crawler. Each AI bot is also classified by purpose, visible in the Categories breakdown: AI training (crawls that feed model training, such as GPTBot and ClaudeBot), AI search (crawls that feed an answer engine’s index, such as OAI-SearchBot and PerplexityBot), and AI agent (on-demand fetches triggered by a person’s prompt, such as ChatGPT-User and Claude-User). An agent fetch is the closest bot signal to a human visit — someone asked the AI about your page. The bot-traffic-over-time chart draws a second “of which AI crawlers” line beneath the total, so you can see AI crawling grow independently of ordinary bot traffic. It’s styled in violet rather than the red of the total line — an AI crawler reading your site is usually something you want if you care about being cited in AI answers.

​

How is bot traffic broken down?

Below the chart, bot traffic is broken down the same way human traffic is:

• Referrers, Pages, Countries — where bots come from and what they hit.
• Devices — the browser, operating system, and device a bot reports (often sparse for headless crawlers).
• Categories, patterns, and networks — the bot’s category (search, AI training, AI search, AI agent, social), the matched pattern, and the operating network.

Click a filterable row to narrow the whole report to it.

​

Crawls vs referred visits, per operator

For API users, GET /api/sites/:id/bots/ai-summary answers “who crawls me versus who sends me visitors” in one call: per AI operator (OpenAI, Anthropic, Perplexity, and others) it returns AI crawler requests next to the sessions that operator’s assistant referred to your site over the same window — the two halves of the AI loop joined under one name.

​

Frequently asked questions

​

Related