See which AI crawlers visit your site, serve llms.txt automatically, and manage AI bot access. Everything stays in your own database.
Alternatives to Block Common Crawl via robots.txt
Plugins below are selected for functional relevance and current directory signals. Active-install counts do not affect this ranking.
Relevant replacements
WordPress plugins with similar use cases
Compatibility and update metadata provide context, not a quality guarantee. Review each plugin’s details for your site.
Blocks the archive.org bots through the WordPress virtual robots.txt file.
Block GPTBot, ClaudeBot, Bytespider and 170+ AI crawlers and scrapers, bot by bot. Blocks by user agent and robots.txt. Optional daily list updates.
Measure detected AI crawler activity, see which content is accessed, identify AI referral visits, and manage bot access locally.
See which AI crawlers request pages on your site, choose what each one is told, and check whether it took any notice.
Replace the default WordPress robots.txt workflow with a smarter, structured version you can preview before publishing, with Free, Pro, and Premium ed …
See which AI crawlers read your site — GPTBot, ClaudeBot, PerplexityBot and more — verify they are who they claim, and block the ones you choose.
Monitor identifiable AI crawler traffic and enforce access rules site-wide, per crawler, or per post, with optional llms.txt publishing.
Choose which AI bots can read your content, and publish an llms.txt file for AI search tools.
Can GPTBot, ClaudeBot, PerplexityBot and Googlebot reach your site? robots.txt per bot, a live test, a bot visit log and a weekly email alert.