About this plugin
txt Generator is a lightweight, high-performance plugin that automatically compiles and serves structured markdown index directories for LLM agents, scrapers, and web crawlers.
txt directory index at your root domain, search crawlers can easily list, parse, and consume your content. txt which concatenates full page markdown for accelerated ingestion.
txt Generator features native deep integration with WooCommerce. It reads product specifications and converts catalog information (SKU, price, stock status, categories, and descriptions) into clean, standard-compliant markdown for seamless AI catalog indexing.
Engineered for Performance & Compatibility
This plugin is built from the ground up with a focus on e-commerce catalog complexity, resource efficiency, and compatibility, ensuring it uses minimal server resources:
- Transient Database Caching: Directory and content pages are cached for 12 hours using WordPress Transients. When an AI crawler requests the directory, the cached markdown is served instantly without querying the database, resulting in near-zero CPU and RAM overhead.
- Smart Cache Purging: Caches are automatically flushed whenever you create, edit, trash, or delete posts, pages, or WooCommerce products, keeping your indexes updated without manual work.
- Size Protection Limits: Caps query counts per post type to prevent memory exhaustion on larger websites.
- Conflict-Free Architecture: All classes, functions, options, and hooks use isolated namespace prefixes (like sali_llms_ ), ensuring complete compatibility with core WordPress features, SEO plugins, and third-party tools.
Key Features
- WooCommerce Catalog Integration: Translates products, prices, SKUs, stock status, and descriptions into clean markdown.
- Dynamic Search Retrieval API: Real-time search query endpoint ( /llms-search.txt?q=... ) to allow conversational AI agents to search your catalog and posts instantly.
- Selective AI Bot Blocker: A configuration grid to block specific crawlers with a 403 Forbidden header and inject custom Disallow directives into the dynamic robots.txt output.
- AI Crawler Analytics Tracker: An admin dashboard tracking cumulative and daily crawling logs of major AI agents.
- Server-Agnostic File Generation: A single-click “Generate llms” button that writes physical files to the site root using standard WordPress Filesystem APIs, resolving Nginx 404 issues without server edits.
- Robots.txt Auto-Integration: Automatically appends the directory link to your site’s dynamic robots.txt file for native AI discovery.
- Dynamic Custom Post Types: Scans all registered public post types (Posts, Pages, WooCommerce Products, Portfolios, etc.) dynamically for index inclusion.
- Granular Exclusions: Exclude specific items by Post IDs or entire post/product categories by comma-separated slugs.
- Custom AI Instructions: Prepend a system prompt to guide crawlers on how to consume your site’s content.