AJ Agent Crawl Optimizer – AI Readiness Scanner & llms.txt for WordPress

Run the built-in readiness scanner. AJ Agent Crawl Optimizer checks your live site against 33 AI-agent-readiness checks — Cloudflare’s 21 isitagentready.com checks plus 12 modelled on what ora.ai / is-agentic.com grade for — scores you 0–100 with a grade and a Level 0–5 ladder from Not Ready to Agent-Native. Every verdict shows the real request and response behind it, and failing checks get a one-click Fix now button that re-scans to prove it passed.

llms.txt is a plain-Markdown file at /llms.txt that gives AI agents a curated index of your most important content, proposed at llmstxt.org. AJ Agent Crawl Optimizer lets you hand-curate it: write the intro, pick which post types (and WooCommerce products) appear, and set a per-post “Summary for AI agents” that overrides the excerpt. It is a surface AI agents can route on — not a guaranteed path to AI citations, but a machine-readable map of what your site is for.

The plugin writes explicit robots.txt rules for 20 AI crawlers — GPTBot, ChatGPT-User and OAI-SearchBot (OpenAI), ClaudeBot, Claude-User and Claude-Web (Anthropic), PerplexityBot and Perplexity-User, Google-Extended, CCBot and Bytespider among them — each allowed or blocked per a per-bot policy. Presets like “Allow search & user requests, block training” make it a single choice. Remember: to be cited by an AI engine you must allow its crawler.

There is no guaranteed path, but three things help and this plugin does all three: allow the AI search crawlers (OAI-SearchBot, PerplexityBot, Google-Extended), publish the machine-readable surfaces agents look for (llms.txt, JSON-LD, Markdown negotiation), and write a clear “Summary for AI agents” on key pages. The readiness scanner then verifies each of these is actually live over real HTTP — so you fix what’s broken instead of guessing.

/llms.txt is a curated index — a Markdown list of your key pages with links and short summaries, so an agent can decide what to fetch. /llms-full.txt serves the full Markdown content of those same curated entries in one file, for agents that want everything inline. The plugin serves both, and password-protected content is always excluded from each.

Most plugins only generate an llms.txt file from fixed rules. AJ Agent Crawl Optimizer is an agent-readiness scanner and fixer: it audits your live site over real HTTP against 33 checks, scores it 0–100 with a grade and a Level 0–5 ladder, shows the request/response evidence behind every verdict, fixes failures in one click, and re-verifies. It covers the answer engine optimization (AEO) and generative engine optimization (GEO) groundwork — structured data, AI crawler rules, and MCP discovery — that plain generators and content-writing SEO plugins skip.

Yes. WooCommerce products are detected automatically and can appear as their own section in your curated llms.txt, so AI agents can see your catalogue. You control whether products appear, how many, and in what order, and you can exclude individual products or give any product a per-product “Summary for AI agents.”

Dashboard is the verifier: it scans your live site over real HTTP, shows your score, grade and Level 0–5 with evidence for every check, and fixes failures with one click. Settings is the switchboard: the per-feature toggles, wizard, and testing tools. A toggle can be ON in Settings while the Dashboard still shows a fail — that gap usually means a page cache or server rule is intercepting the endpoint, and the evidence view shows exactly what happened.

The scanner runs Cloudflare’s 21 isitagentready.com checks plus 12 checks modelled on the ora.ai / is-agentic.com catalog and the AgentReady.org standard. The 0–100 score weights every check; checks that don’t apply or can’t be reached drop out of the denominator, bonus checks (Web Bot Auth, DNS-AID, A2A card, commerce protocols) add up to 5 points and never subtract, and grades use the ora.ai bands (A+ 95, A 86, B 70, C 48, D 28). The Level 0–5 ladder stays identical to Cloudflare’s: Level 1 needs 2 of robots.txt/sitemap/Link headers; Level 2 adds AI bot rules and Content Signals; Level 3 adds Markdown negotiation; Level 4 adds one of MCP card/Agent Skills/API catalog/A2A card; Level 5 adds two of Web Bot Auth, all four integrations, and auth metadata. Nothing counts unless the scan verifies it — enabling a toggle earns nothing until the endpoint actually responds. Commerce checks are informational and never scored.

The plugin never creates files: every endpoint (/.well-known/*, /llms.txt, /openapi.json, /auth.md, /agent-ready-badge.svg) is served virtually by WordPress, so hosts that forbid file creation are fully supported. What can break things is the web server intercepting requests before WordPress runs — typically an nginx dot-path deny rule returning 403 on /.well-known/*, or a static-file rule returning 404 for /llms.txt-style paths. The readiness scan detects exactly this (feature enabled + server blocking) and the Dashboard shows a hosting notice with copy-paste nginx and Apache fixes you can apply yourself or forward to your hosting support.

Yes — it is curated, not auto-generated from fixed rules. Under Agent Ready → llms.txt you write the intro, pick which post types appear (custom post types and WooCommerce products show up automatically), and set each section’s heading, item count and order, plus your own Markdown block. A live preview shows the file as you edit, before you save.

Individual entries are steered from the editor: the “Agent Ready (llms.txt)” panel has an Include in llms.txt toggle (excluded pages stay out of both /llms.txt and /llms-full.txt ) and a Summary for AI agents field that overrides the excerpt for that entry — write it for a model deciding whether to fetch the page. The defaults keep the same sections and ordering as before, so upgrading needs no attention until you decide to curate.

Yes: wp agent-ready scan (add --format=json for machines or --format=agent for a markdown fix report), wp agent-ready fix <check> or --all-safe , and wp agent-ready status . Add --min-score=<n> or --min-level=<n> to scan or status for a CI gate that exits non-zero. The same operations are available over REST at /wp-json/ajaco/v1/scan , /wp-json/ajaco/v1/fix and /wp-json/ajaco/v1/external (admin capability required; /wp-json/ajaco/v1/health is public).

No. JSON-LD and the identity meta tags auto-suppress when Yoast SEO, Rank Math, All in One SEO, SEOPress, The SEO Framework, Slim SEO, Squirrly SEO, Schema Pro, or SASWP is active. The settings page shows a clear notice when this happens. Our robots_txt filter runs at PHP_INT_MAX priority, so any rules your SEO plugin adds (sitemap URLs, custom Disallow, etc.) are preserved — our Content-Signal line is appended last.

No. Browsers send Accept: text/html and get the normal HTML response. The .well-known/... , /llms.txt , and /openapi.json endpoints are paths a normal user never visits. What the enabled features add to a page is invisible to visitors: a JSON-LD <script> block and Open Graph meta tags (when no SEO plugin is active), a few <link rel> tags and Link headers pointing at the agent-facing files, and a small WebMCP script tag that no-ops in browsers without the API.

Yes. Every endpoint resolves at both the root and per-subsite paths automatically — /llms.txt , /blog/llms.txt , /.well-known/api-catalog , /blog/.well-known/api-catalog , etc. Each subsite has its own settings.

Use the ajaco_required_capability filter. The plugin defaults to manage_options (admin-only); change it to any capability of your choice, e.g. edit_pages for editors:

add_filter( 'ajaco_required_capability', function () { return 'edit_pages'; } );

Check three things: (1) the IndexNow API Key field is filled in, (2) you’re publishing a post of a public post type — revisions and autosaves are skipped, (3) you’re on production. Pings are non-blocking, so failures are silent. Tail your server log for outbound requests to api.indexnow.org .

Yes. The plugin exposes nineteen filter hooks. Examples:

add_filter( 'ajaco_required_capability', function () { return 'edit_posts'; } );

Delegate plugin access to a non-admin role.

add_filter( 'ajaco_skill_definitions', function ( $skills ) { return $skills + [ 'products' => [ 'type' => 'information-retrieval', 'description' => 'WooCommerce products', 'endpoint' => rest_url( 'wc/v3/products' ) ] ]; } );

Register custom skills that ship in the Agent Skills Index as SKILL.md artifacts with verifiable sha256 digests.

add_filter( 'ajaco_content_signal', function () { return 'ai-train=yes, search=yes, ai-input=yes'; } );

Customize the Content-Signal directive (e.g. permit AI training).

Other hooks: ajaco_json_ld_graph (add Product, Recipe, Event… schema), ajaco_llms_txt_content and ajaco_llms_full_txt_content (append to or replace the files), ajaco_llms_post_types , ajaco_llms_exclude_post , ajaco_openapi_spec , ajaco_api_catalog_linkset , ajaco_ai_catalog , ajaco_mcp_server_card , ajaco_mcp_endpoint (declare an MCP server the plugin cannot det