It Was Not AI That Came, It Was Only ChatGPT
Memory and traffic jumped and analytics showed nothing. Counting the server log by User-Agent, 11,321 of 15,001 requests over three hours were one backlink-tool bot. After blocking seven bots I looked at analytics again, and the "AI Assistant, 53 sessions" bucket turned out to be one company
The disclosure-summary frontend runs on Railway. One night its memory and hourly egress both jumped, even though the last deploy had been two days earlier and analytics showed no change at all. Traffic was up and visitors were not.
What I set out to do
Railway usage jumped overnight. The disclosure-summary frontend went from 0.35GB of memory to 0.65~0.9GB, and hourly egress from 0.02GB to 0.11GB. The last deploy was two days earlier, so this had nothing to do with a code change.
That evening I asked.
since yesterday dartbrief-web’s memory and network usage on railway went up.. but GA shows nothing at all in realtime.. what is this?? SEO maybe??
Traffic jumped and analytics has nothing. That combination was the hint. Analytics only records a session when a script runs in a browser. Visitors that do not run scripts are entirely invisible.
How I asked
So the thing to look at was not analytics but the requests the server actually received. I had it pull three hours of Railway HTTP logs and group them by User-Agent.
15,001 requests split like this.
| User-Agent | Requests |
|---|---|
| AhrefsBot | 11,321 |
| meta-externalagent | 1,607 |
| everything that looks human | 479 |
| SemrushBot | 350 |
| MJ12bot | 150 |
| PetalBot | 107 |
| Bytespider | 78 |
| DotBot | 1 |
One backlink-research tool’s bot was 75% of everything. Humans were 3%. No wonder analytics showed nothing. Ninety-seven percent of the visitors were on the side that does not run scripts.
Looking at the numbers I asked one thing.
if AhrefsBot crawls me what do I get out of it?
The answer was nothing. We are not the ones subscribing to that tool. It scrapes our site to build someone else’s report, and we pay for it.
should we just block all the crawlers that do not actually help real user traffic?
Where I got stuck
After the block list was drafted, something nagged at me, so I asked for one more thing.
and look at the actual google analytics data and judge one more time.. we barely have any data anyway..
For the record, the blocking commit landed at 21:15 and this re-measurement at 21:27. I blocked first and measured twelve minutes later. The order was backwards.
And that twelve-minutes-late measurement caught something else. Not the block list, but the metric I had been reading all along.
The 28-day channel rollup looked like this.
| Channel | Sessions |
|---|---|
| Direct | 114 |
| AI Assistant | 53 |
| Cross-network | 13 |
| Organic Search | 6 |
Nine times more AI than search. Looking at that screen alone, the only available conclusion is that AI referral is now a major channel. That is what I believed.
Pulling 90 days by source instead broke the bucket open.
| Source | Sessions |
|---|---|
| direct | 144 |
| chatgpt.com | 90 |
| bing organic | 6 |
| copilot.com | 2 |
| google organic | 0 |
| Meta AI · Perplexity · Claude | 0 |
The contents of the single cell labeled “AI Assistant” were effectively one company. Copilot had two. The rest had not sent a single session. Google search referral was zero across 90 days. There were 448 impressions and zero clicks.
One choice of aggregation unit was rewriting the whole judgment. By channel it reads “AI in general is coming.” By source it reads “only ChatGPT is coming.” The first sentence makes blanket-blocking AI crawlers sound natural. The second makes it suicide. Block ChatGPT’s search-index crawler and the one that fetches live when a user asks, and you sever nearly the only path by which a human reaches this service.
How I worked it out
The measurement did not change the block list. All seven blocked bots contributed zero referral, and that stayed true after measuring. What changed was the criterion.
My first criterion was “block anything with zero analytics sessions.” That criterion is wrong. Google’s search bot also has zero sessions. Bots do not run scripts, so of course it is zero. By that rule I would have to block the search bot too.
The revised criterion is “does it create a path a human can follow?” Does it get you into search results, attach you as a source on an answer, or build a share card? Do any one of those and you stay. Scrape and nothing else and you go.
Under that criterion, crawlers from the same company split. The one that scrapes only for training is blocked, and the one that fetches live when a user drops a link stays, because there is a human behind it. Same with backlink tools. The bot scraping for someone else’s report is blocked, and the one that returns an audit of our own site stays.
Then two more things came out of the tests.
First, the test was case-sensitive. User-Agent matching in robots.txt is not. But the verification test was comparing strings as written. Write gptbot in lowercase in the list and GPTBot gets blocked wholesale in reality, while the test reports it is not in the block list. Mistype one line and you sever a referral path while looking at a green check.
Second, the way blocks were counted was loose. Every group that was not the catch-all was being counted as a block. That means the disallow rule could be deleted entirely and the test would still pass, and a group that carves out an exception for one bot would be misread as a block. I changed it to count only groups whose disallow list contains the root.
Finally I wrote the limits into the commit. All 90 days is 250 sessions. What this data supports is “the things I blocked have no path to send a human,” and nothing as general as “AI assistants are strategically the second channel.” Undoing it costs one line off a list, which is why I took the risk.
What I learned
- The traffic analytics cannot see may be most of your traffic. Humans were 3%. None of it was visible until I counted the requests the server received.
- One aggregation unit flips the conclusion. Bucketed by channel it is “AI is coming,” broken out by source it is “only ChatGPT is coming.” Always open the bucket once.
- Measure before you block. I was twelve minutes late and the list happened not to change, but if it had, it was already committed.
- Do not use a metric as a criterion. “Zero sessions means block” blocks Google’s search bot. The criterion has to be “is there a path a human can follow.”
- Match the test’s comparison rules to the real matching rules. Verify a case-insensitive rule with a case-sensitive test and the green check is lying.
- If the sample is small, write that it is small. You cannot talk channel strategy off 250 sessions. I only took it because undoing it is cheap.