LLMs.txt Curator

llms.txt is a proposed standard (llmstxt.org) that provides AI systems with a curated, Markdown-formatted overview of a website’s most important content. A strategic selection of the pages you want AI to know about, not a sitemap.

Yes, read only, and only when signed in as a user who can manage Curator. Curator registers abilities with the WordPress Abilities API (WordPress 6.9 or later): get-site, list-resources, get-resource, get-representation, get-changes, get-published-file, explain-resource and get-diagnostics. They are available through PHP and the REST API. If you install WordPress’s MCP Adapter plugin, AI clients that speak MCP can use four of them: get-site, list-resources, get-representation and get-changes. Curator does not include an MCP server of its own.

“Public” in an ability’s settings means listed for tools; it never means anyone can call it. Every call checks the signed-in user. To keep Curator’s abilities out of MCP lists, return false from the llmscu_abilities_public filter.

The abilities answer only from what you publish. A draft, private, password-protected or excluded page gets the same “not available” answer as an address that does not exist. A page’s Markdown is returned only if a logged-out visitor can read the page now, and only if you publish Markdown (Markdown pages or llms-full.txt). Calling them never changes your settings, curation or what you publish, and Curator takes no snapshots and makes no requests to answer them. (When a page’s Markdown comes from its stored content, its blocks are rendered as they are when the page is viewed, and a block that fetches something, such as an RSS block, still does.)

get-changes counts publishes, not time. Each publish has a number; a tool keeps the number it was given and asks what was added, changed or removed since. Pages whose content changed without their llms.txt line changing have a new fingerprint in list-resources, so a tool can tell what to fetch again.

Page text in a returned Markdown version is written by your site’s authors and visitors, and tools should treat it as content, not instructions.

A widely used companion to llms.txt , rather than part of the llms.txt proposal itself. While llms.txt contains links and short descriptions, llms-full.txt contains the full Markdown content of each page. Optional: enable it in Settings when ready.

No. llms.txt alone is sufficient. llms-full.txt is useful if you want AI systems to have immediate access to your full content without additional crawling.

Scans every curated page and fills missing descriptions using a five-step fallback chain: schema markup -> SEO meta -> excerpt -> Open Graph -> page content. Pages with descriptions already set are never touched.

A count of how many listed pages have descriptions, shown in the admin with the pages that still need one. It is not published in the file.

Lets you set a different title for a page in your llms.txt output without changing it on your site. Useful when your WordPress title includes your site name but you want AI to see a cleaner, more descriptive title.

They run before every publish. If errors are found, the file is not published and the results are shown immediately, so a broken or malformed file never goes live. Switch them on or off in Settings.

Rank Math, Yoast SEO, All in One SEO, SEOPress, and The SEO Framework.

GPTBot (OpenAI), ChatGPT-User, ClaudeBot (Anthropic), PerplexityBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent, Bytespider (ByteDance), CCBot (Common Crawl), Cohere, DeepSeek, and Amazonbot.

The plugin has a rewrite rule fallback that serves both files via WordPress. Choose between “Direct file”, “Rewrite rule only”, or “Both” (recommended) in Settings.

Yes. Activate network-wide from Network Admin > Plugins, or activate per-site on individual sub-sites.

Each site manages its own independent llms.txt ; there is no shared network file. The Network Admin overview page (Network Admin > Settings > LLMs.txt Curator) shows every site’s generation status and lets you regenerate any site, or all sites at once, without leaving the network admin.

The main site writes a physical file. Sub-sites, on both subdirectory networks ( example.com/site1 ) and subdomain networks ( site1.example.com ), share the main site’s document root, so they serve llms.txt via WordPress rewrite rule from the database instead. This is correct and functionally identical.

WP-CLI works per-site using the standard --url= flag: wp llms-txt regenerate --url=https://site1.example.com

Password-protected pages are always left out, and so are pages your membership plugin restricts. Both files are generated as a logged-out visitor, so members-only sections inside a public page are left out too. Pages your SEO plugin marks as noindex are left out while “Leave out noindex pages” is ticked (the default). Items whose content type is not ticked under Content Types are left out too, and the validator lists each one so nothing disappears silently.

Rank Math, Yoast SEO and All in One SEO can generate llms.txt too. This plugin never overwrites a physical llms.txt it did not create. If it finds one, the status panel says so and offers “Replace existing file”, which writes and checks the new file first, then moves the old one to a protected backup folder in uploads (with a random file name). If anything fails part-way, the original is put back. Switch the llms.txt feature off in one of the two plugins so they do not compete.

With Markdown pages on, Curator requests each page in your llms.txt from your own site, as a logged-out visitor with no cookies, in the background. It uses the page’s main content area, leaves out menus, headers, footers, comments and share buttons, and converts what remains. If a snapshot cannot be taken (a password-protected staging site, a maintenance page, a host that cannot answer its own requests), the page’s stored content is used instead and the Markdown page says which. A snapshot is never used after its page has changed.

Only with Markdown pages on, only for pages in your llms.txt, and only in the background: never while a visitor is waiting. Up to 20 pages are requested at a time, a minute or so apart, until all have a snapshot; then each is renewed about once a day, or sooner when the page changes. If your site cannot answer its own requests, Curator stops trying and waits longer each time. With llms-full.txt on and Instant publishing, llms-full.txt is rebuilt after new snapshots; on Daily, Weekly or Manual it changes only when you publish.

Yes, with the llmscu_extraction_noise filter. Each rule names a tag, id, class or attribute to remove from the content area. The Inspector’s Diagnostics view lists what was removed from each page.

No. Everything stays on your server. No telemetry, no external API calls, no cookies. Crawler IP addresses are anonymised before storage.