ScrapeLock – Content & AI Bot Protection

Most content protection plugins disable right click in the browser. Scrapers and AI crawlers never use a browser, so they are not affected. ScrapeLock works on the server, where automated copying actually happens, and keeps real visitors, customers and search engines unaffected.

What makes ScrapeLock different

• Real search engines vs. fakes. Googlebot, Bingbot, Applebot and Yandex are verified against their official IP lists and with reverse + forward DNS. Real crawlers are never limited or blocked; scrapers that only pretend to be Googlebot get a 403.

• AI training vs. AI search. Training crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended, Bytespider and more) are blocked, while AI search assistants (OAI-SearchBot, ChatGPT-User, PerplexityBot and more) can be allowed with one option, so your site still appears in AI answers with a link.

• Per-IP limits and temporary blocks. Separate limits for pages, searches and REST / GraphQL requests. IPs that keep exceeding them are blocked for a while, optionally also at your CDN, and an invisible trap link catches bots that ignore robots.txt.

• Private feeds with a token. XML feeds for price comparison sites, Google Merchant or an ERP are closed to everyone else. The token goes in a header or the URL, only its hash is stored, and it can be rotated with a grace period.

• Your text stays out of the API. Post, page and product text is hidden from the REST API, WPGraphQL and full-text RSS for visitors, which are the routes scrapers use instead of the HTML.

• Rules for your server and CDN. Ready-made Nginx, Apache and Cloudflare rules, generated for your own site and settings, so pages served from cache are protected too.

• Emergency Mode. One click during a scraping attack: stricter limits for 1 hour to 3 days, then your normal settings return on their own.

Nothing can stop a determined human: whatever a visitor sees has already reached their device. ScrapeLock closes the automated routes and leaves traces: a block log, watermarks, a source link on copied text and a machine-readable rights reservation (TDM).

Bots and limits

• Block a list of scrapers and AI training crawlers, with a separate option to allow AI search assistants.

• Verify allowed AI search bots (OpenAI OAI-SearchBot and ChatGPT-User, PerplexityBot, Perplexity-User) with their official IP lists: real ones are never limited, fakes that only use their name are blocked.

• Detect fake Googlebot, Bingbot, Applebot and Yandex with the official IP lists and reverse DNS. Real search engines always pass.

• Invisible bot trap link, disallowed in robots.txt.

• Optional blocking of headless browsers that reveal themselves in their headers, and of scripts that copy a browser’s User-Agent but not the headers a real browser sends. A carefully disguised automated browser is not detected this way; the page limits, the human check and the bot trap still apply to it.

• Per-IP limits for pages, searches and REST / GraphQL (IPv6: per /64 network), with temporary blocks after repeated violations.

• Optional blocking at the CDN: temporary blocks are also applied by Cloudflare or Bunny CDN, so a persistent scraper never reaches your server. Optionally, only requests that come through the CDN are answered.

• A separate limit for “page not found” (404) requests, for scrapers that guess URLs. Missing images, CSS and scripts do not count.

• Optional human check: visitors who have not interacted with the page (scroll, tap, key) get a stricter page limit. No captcha, nothing to click; it works with page caches.

• Optional sitemap limit (WordPress, Rank Math, Yoast and similar sitemaps), never for verified search engines.

• Optional country rules: blocked countries, or a stricter page limit for countries your visitors do not come from. Uses Cloudflare’s or Bunny CDN’s country header, or the free DB-IP Lite database.

• Optional pre-cache check: the page limit and the blocks also apply to pages served from a page cache such as WP Rocket, through one line that ScrapeLock adds to wp-config.php and removes again.

• Always-blocked lists of IPs / CIDR and referrer domains.

• Hide usernames from visitors: no?author=N redirect, no REST user list, no users sitemap.

• Optional early check: a small must-use plugin that stops known bots and blocked IPs before the theme and the other plugins load.

• Content Signals in robots.txt (search yes, AI training no).

• Trusted Proxy support for Cloudflare, Bunny CDN and reverse proxies, so the real IP cannot be spoofed. Cloudflare and Bunny CDN are detected, with a one-click fix when every visitor appears with a CDN address.

• Works with the Cloudflare plugin (APO): blocks and private feeds are never cached at the edge, and the Cloudflare cache is purged when you save the settings.

• Ready-made presets (Balanced, Strict, Maximum) and an Emergency Mode for attacks in progress.

• Block log with a 24-hour summary, top IPs and User-Agents, the pages scrapers target most, and automatic deletion after 1, 7 or 30 days.

Feeds and API

• Hide post, page and product text from the REST API and WPGraphQL for visitors.

• RSS with a short excerpt only, or disabled. With full text, an optional “The post … first appeared on …” line links republished copies back to the original.

• Private XML feeds (Skroutz, BestPrice, Google Merchant, ERP) protected with a token in a header or URL. Only the token hash is stored. Token rotation with a grace period and per-feed statistics.

• Optional: hide the REST API map (/wp-json/ and the links to it) from visitors, and turn off pingbacks over XML-RPC. A list shows the REST routes on your site that anyone can call without logging in.

• Hide the versions of WordPress, PHP and the plugins: no generator tags, no X-Powered-By, a code instead of “?ver=7.1.3”, and readme / changelog files refused by the server rules.

• Optional: excerpts instead of the full text on lists of articles, an IPv6 limit per /56 network, and blocking of web archives (Wayback Machine).

• With “Hide usernames” on, oEmbed shows the site’s name and address instead of the author’s username.

Site Health, WP-CLI and AI agents

• Tools → Site Health warns about settings that do not work as intended (protection off, every visitor behind a CDN address, a check that is on but not running, Lockdown off from the server) and has a ScrapeLock section in the Info tab.

• WP-CLI: wp scrapelock status, block <ip>, unblock <ip>, blocked, emergency on|off|status and log [--top].

• Abilities API (WordPress 6.9+): scrapelock/get-summary, scrapelock/block-ip, scrapelock/unblock-ip and scrapelock/emergency-mode, for administrators only, so an AI agent or MCP client can read the summary and act during an attack. None of them unlocks Lockdown.

Server rules

Generated Nginx, Apache and Cloudflare rules for your own site, so cached pages and static files are protected too.

On the page

• Block text selection, right click, copy, drag and common shortcuts, or allow copying with the source link appended.

• Printing with the page link at the bottom, or no printing.

• Optional page blur on screenshot shortcuts.

• No framing by other sites (X-Frame-Options, CSP frame-ancestors).

• AI opt-out declaration (TDM): header, meta, /.well-known/tdmrep.json, noai.

Images

• Watermark in a corner, the centre or repeated diagonally, optionally on the full-size image too. The clean original is kept, so the watermark can always be removed.

• Product images are never watermarked by default, because Google Merchant Center and price comparison platforms reject them.

• Batch tool for existing images.

Lockdown

• Freeze the whole site with one click and a code from your authenticator app (Google Authenticator, Microsoft Authenticator, Authy and the like). While it is locked, nobody can change the site from wp-admin, the REST API or XML-RPC, not even an administrator or someone who stole a password.

• Frozen: installing, updating, deleting, activating and deactivating plugins and themes; WordPress core and automatic updates; the file editor; new users, role changes, new administrators; staff email and password changes and resets; new application passwords; all settings; posts, pages, products, media and menus.

• Also enforced against code: if another plugin is exploited, it cannot activate plugins, change the site address or the admin email, add an administrator or unlock the lock.

• Visitors and customers are not affected: they browse, order, pay, register as customers and comment as usual.

• File integrity: when you lock, ScrapeLock keeps a fingerprint of every PHP,.htaccess and.user.ini file (WordPress, plugins, themes, uploads, wp-config.php) and checks it every hour. A new or changed file while locked (usually a backdoor uploaded through a vulnerable plugin) is emailed to you, logged and shown in Site Health. WordPress and the plugins from WordPress.org are also compared with their official checksums, which finds changes made before you locked. Optional quarantine deactivates the affected plugin and renames new PHP files in the uploads folder.

• Sealed: the lock is signed with the keys in wp-config.php and copied into a must-use plugin, so SQL injection in another plugin cannot unlock the site. Settings changed in the database are put back.

• Unlock for 15 minutes to 2 hours, for yourself only (it locks again by itself) or until you lock it again. Locking and unlocking are logged and emailed to the site’s admin address.

• Lost your phone: ten one-time recovery codes, or define( 'SCRAPELOCK_UNLOCK', true ); in wp-config.php as a last resort.

• The QR code is drawn in your browser and the secret is stored encrypted with the keys in wp-config.php: it is never sent anywhere.

• A Lockdown tab in Settings → ScrapeLock, with a full report of what is frozen, what keeps working and which accounts it applies to.

Forms, cart, checkout, the WooCommerce Store API and payment webhooks are not affected.

External services

ScrapeLock downloads public lists of IP addresses, only for the features that use them. Each download is a plain GET request for a public file, made from your server by WordPress (User-Agent “ScrapeLock/version”). No data about your site, your content or your visitors is sent, and the request does not include your site address; as with any download, the provider sees your server’s IP address.

The lists are stored in your database (or, for the country database, in uploads/scrapelock-geo) and checked on your server. They are data, not code or assets: no script, style, font or image is ever loaded from these addresses.

• Google crawler IP ranges, to tell real Googlebot from fakes. Downloaded once a day while protection is enabled, from https://developers.google.com/static/search/apis/ipranges/googlebot.json, https://developers.google.com/static/search/apis/ipranges/special-crawlers.json and https://developers.google.com/static/search/apis/ipranges/user-triggered-fetchers-google.json. Provided by Google LLC: Terms of service, Privacy policy.

• Bing crawler IP ranges, to tell real Bingbot from fakes. Downloaded once a day while protection is enabled, from https://www.bing.com/toolbox/bingbot.json. Provided by Microsoft Corporation: Terms of service, Privacy policy.

• Cloudflare IP ranges, to read the real visitor IP only from Cloudflare. Downloaded once a day only when “Real visitor IP” reads the CF-Connecting-IP or X-Forwarded-For header and “Trust Cloudflare addresses” is on, from https://api.cloudflare.com/client/v4/ips. Provided by Cloudflare, Inc.: Terms of service, Privacy policy.

• Bunny CDN edge server addresses, to read the real visitor IP and country only from Bunny CDN. Downloaded once a day only when “Trust Bunny CDN addresses” is on and “Real visitor IP” reads the X-Real-IP or X-Forwarded-For header, or country rules are set, from https://bunnycdn.com/api/system/edgeserverlist and https://bunnycdn.com/api/system/edgeserverlist/IPv6. Provided by BunnyWay d.o.o. (bunny.net): Terms of service, Privacy policy.

• OpenAI bot IP ranges (OAI-SearchBot, ChatGPT-User), to tell the real bots from fakes. Downloaded once a day only when “Allow AI search engines and assistants” and “Verify AI search bots” are on, from https://openai.com/searchbot.json and https://openai.com/chatgpt-user.json. Provided by OpenAI: Terms of use, Privacy policy.

• Perplexity bot IP ranges (PerplexityBot, Perplexity-User), to tell the real bots from fakes. Downloaded once a day only when “Allow AI search engines and assistants” and “Verify AI search bots” are on, from https://www.perplexity.com/perplexitybot.json and https://www.perplexity.com/perplexity-user.json. Provided by Perplexity AI, Inc.: Terms of service, Privacy policy.

• Anthropic bot IP ranges (Claude-SearchBot, Claude-User), to tell the real bots from fakes. Downloaded once a day only when “Allow AI search engines and assistants” and “Verify AI search bots” are on, from https://claude.com/crawling/bots.json. Provided by Anthropic PBC: Terms of service, Privacy policy.

• DB-IP Lite country database, to apply the country rules. Downloaded once a month only when “Download the free country database” is on, from https://download.db-ip.com/free/dbip-country-lite-YYYY-MM.csv.gz (the current or the previous month). Lookups happen on your server. Licensed under CC BY 4.0 (“IP Geolocation by DB-IP”). Provided by db-ip.com: Terms of service, Privacy policy, License.

The five data center lists below are used only for the “Data center limit” (Bots and limits), a stricter page limit for visitors that come from cloud servers rather than from home or mobile connections. Each one is downloaded once a day only while protection is enabled and “Data center limit” is above 0 (set it to 0 to turn the lists off). They are JSON or CSV data files fetched by your server with wp_remote_get(); nothing from these hosts is ever loaded in your visitors’ browsers.

• Amazon Web Services IP ranges (only the EC2 servers are used), from https://ip-ranges.amazonaws.com/ip-ranges.json. Provided by Amazon Web Services, Inc.: Terms, Privacy policy.

• Google Cloud IP ranges, from https://www.gstatic.com/ipranges/cloud.json, the address where Google publishes the list. Provided by Google LLC: Terms of service, Privacy policy.

• Oracle Cloud IP ranges, from https://docs.oracle.com/en-us/iaas/tools/public_ip_ranges.json. Provided by Oracle Corporation: Terms of use, Privacy policy.

• DigitalOcean IP ranges, from https://digitalocean.com/geo/google.csv. Provided by DigitalOcean, LLC: Terms of service, Privacy policy.

• Akamai Cloud (Linode) IP ranges, from https://geoip.linode.com/. Provided by Akamai Technologies, Inc.: Terms of service, Privacy policy.

org with their official checksums. org services WordPress itself uses for updates. json); no other data about your site, your content or your visitors is sent.

org: Privacy policy.

Optional blocking at the CDN (off by default) uses the API of the CDN you choose, with your token. It sends only the IPs ScrapeLock blocks temporarily (IPv4, IPv6 or an IPv6 /64), each with the reason and end time, and removes them when the block ends. Saving the setting makes one read-only request to check the token.

• Cloudflare API (IP Access rules of your zone), only when the CDN is set to Cloudflare: https://api.cloudflare.com/client/v4/zones/ZONE_ID/firewall/access_rules/rules. Provided by Cloudflare, Inc.: Terms of service, Privacy policy.

-

net/pullzone/PULL_ZONE_ID (addBlockedIp, removeBlockedIp, and reading the zone’s list of blocked IPs). o. net): Terms of service, Privacy policy.

…