99% of My Website Traffic Is Bots

By PatronView | Published August 7, 2026

👋 That's me waving! (I'm doing it because blue is a favorite color of mine).

In the spirit of transparency, I'm going to reveal the exact firewall configurations I use. This way, if you're a bot developer, you can figure out how to bypass them more efficiently. Honestly, I'm not even sure if a web crawler reads blog posts to find "how-to" guides on avoiding detection.

For the past year, I've been locked in a digital war against scrapers on my philanthropy donor database, PatronView. When you're "vibe coding," nobody warns you about this side of the internet, so I'm documenting my findings.

The Scale of the Invasion

To give you an idea of the sheer volume:

The China Spike: In a single 24-hour window, my site was hammered by 3.6 million requests from Chinese botnets.
The Claude Disparity: According to Cloudflare, Anthropic's crawlers hit roughly 3,000 pages for every single human visitor they refer.
The Daily Grind: One specific bot was reading 117,000 pages every day without ever sending a single actual person to my site.

The "Invisible" Traffic

In the week surrounding this post, my server processed 2.5 million external requests, serving 1.28 million full pages.

The math looks like this: $\text{Bot-to-Human Ratio} \approx \frac{2.5\text{M requests}}{11.6\text{K human views}} \approx 214:1$

I utilize the Plausible Community Edition (self-hosted) for my analytics. Because Plausible only tracks users who execute JavaScript, my dashboard shows a modest, quiet site with about 500 daily visitors. If you rely solely on JS-based tools like Google Analytics or Fathom, you are completely blind to the actual load hitting your server.

The Nature of the Beast: PatronView

PatronView is a custom-built database of American philanthropists. I constructed it using various intelligence-gathering methods, resulting in 1.5 million individual profile pages derived from:

Annual reports
Public donor walls
IRS 990 forms

~~I am a hypocrite~~ Full disclosure: My site exists because I scraped those public documents. I am a scraper writing a manifesto against scrapers.

The Evolution of the Attack

The onslaught happened in waves:

Phase 1: The SEOs. Early on, bots like SemrushBot, AhrefsBot, MJ12bot, and DataForSEOBot relentlessly crawled every page. Blocking these via robots.txt and Cloudflare security rules was my "gateway drug" into bot mitigation.
Phase 2: The Stealth Crawlers. In November 2025, 4,000 "visitors" appeared. I knew they were bots because they had no referrer and exclusively targeted my "fund pages"—sections that only 10% of actual humans ever visit.

The Day China Arrived

On April 22, the floodgates opened. My site received 3.6 million requests in one day from 361,844 unique IP addresses, almost all originating from China.

Cloudflare's "Managed Challenge" (the invisible CAPTCHA) blocked 1.18 million requests in the first ten hours, but a worrying number of bots were still passing the test.

The Nuclear Option

I did the thing that "best practices" tell you not to do: ~~I kept it open~~ I blocked the entire country of China at the edge.

Shortly after, a similar wave hit from Vietnam, so I blocked them too. Since 95.9% of my organic traffic is from the US and 1.3% from Canada, I have zero interest in allowing access from China or Vietnam.

Community Consensus: When I shared this on Twitter, I found I wasn't alone.

Matt Paulson (MarketBeat): "Don't forget to add Russia to the list."

Jack Ellis (Fathom): "We saw a ton of spam traffic from China... customers have seen China drop completely now."

Rodrigo Rocco: "They are using thousands of residential IPs making only 1 call each, so hard to stop it."

The "Claude Ratio" and AI Parasitism

Cloudflare's AI dashboard revealed a staggering statistic regarding Claude-SearchBot. For every one human visitor referred to my site, the bot crawled 35,000 pages.

The Irony: I used Claude Code to build this website and to write the API rules for Cloudflare that now block Claude. I've made peace with this paradox.

Crawl-to-Referral Metrics

I've developed a new metric to evaluate bots: Pages crawled per visitor referred.

Bot Name	Ratio (Pages:Humans)	Impact
`Googlebot`	$46:1$	Acceptable
`Bingbot`	$406:1$	Moderate
`Claude-SearchBot`	$35,000:1$	Parasitic
`Amzn-SearchBot`	$\infty:1$	Pure Extraction

Currently, Amzn-SearchBot (which powers Alexa and Rufus) is my most active crawler, hitting the site 117,000 times per day. It provides zero traffic, zero newsletter sign-ups, and likely zero attribution.

Final Defense Strategy

To keep the site performant, I use a layered defense:

Cloudflare AI Crawl Control: Automatically blocks declared training bots like GPTBot, ClaudeBot, CCBot, and Bytespider.
Geographic Blocking: Hard blocks on high-bot-volume countries.
Managed Challenges: Forcing suspicious traffic to prove humanity.
Custom 403 Rules: Specifically targeting high-ratio AI crawlers.

// Conceptual logic for a custom firewall rule
if (request.bot == 'Amzn-SearchBot' || request.country == 'CN') {
    return response.status(403).send("Access Denied");
}

By implementing these, I've managed to drop some crawlers from 60,000 requests per day down to a mere 25.