Bravo Crawl Hunter – AI Crawl Blocker, Block AI Bots & Crawlers

🛡️ Two ways to control access

• robots.txt rules – Publishes Disallow rules for the crawlers you select. Well-behaved bots honor them, though not every bot does.
• Hard block (403) – Returns an HTTP 403 response when a selected crawler reaches WordPress, whether or not it follows robots.txt.

The two methods are independent. Use either one, or both together, from Settings → General Settings .

🔗 Covers WordPress content endpoints

Hard blocking applies to front-end requests as well as WordPress REST API and XML-RPC requests, helping prevent selected crawlers from accessing your content through alternate WordPress endpoints.

⚙️ Custom User-Agents

Bots and scrapers that are not in the registry can be blocked by adding their User-Agent string, making it easy to use Bravo Crawl Hunter as a custom bot blocker for unwanted crawlers.

Up to 50 entries are supported, each matched anywhere in the incoming User-Agent header and case-insensitively, and enforced with the same protection methods as the registry.

🚀 Works with caching and CDNs

Server-side detection runs when a request reaches WordPress. A page served entirely from a server cache, caching plugin, or CDN may never reach WordPress, so it cannot be classified that way.

For those requests, the optional JavaScript Detection feature records known crawlers that load a cached page containing the plugin’s lightweight detection script.

It is disabled by default and can be enabled from Settings → General Settings .

📊 Dashboard

The dashboard summarizes crawler activity over the date range you pick:

• Total, blocked, allowed, and unknown requests, plus the number of crawlers seen.
• Activity trends across the selected range.
• A breakdown by individual crawler and by crawler category.
• The pages receiving the most blocked crawler requests.
• Recent crawler activity.

📋 Crawler Logs

The Crawler Logs screen is a searchable, filterable, paginated history of classified requests. Each entry records the time, crawler, path, action, and user agent.

Logging is on by default and can be switched off under Settings → General Settings ; dashboard totals and charts keep updating either way.

Log retention is capped by a maximum entry count under Settings → Advanced , and the oldest entries are removed first.

💾 Export, import, and data on uninstall

Settings can be exported to a JSON file and imported back, to keep a backup or move a configuration to another site, from Settings → Advanced .

The same screen lets you reset everything to the recommended defaults, and choose whether settings, logs, and stats are kept or removed if the plugin is deleted.