Botkibble
AI agents, LLMs, and crawlers have to wade through navigation bars, sidebars, ads, and comment forms to reach the content they want, and every element costs tokens. Cloudflare measured an 80% reduction in token usage when converting a blog post from HTML to Markdown (16,180 tokens down to 3,150). Bo
AI agents, LLMs, and crawlers have to wade through navigation bars, sidebars, ads, and comment forms to reach the content they want, and every element costs tokens. Cloudflare measured an 80% reduction in token usage when converting a blog post from HTML to Markdown (16,180 tokens down to 3,150).
Botkibble adds a Markdown endpoint to every published post and page.
Cloudflare offers Markdown for Agents at the CDN edge on Pro, Business, and Enterprise plans. Botkibble does the same thing (for free) at the origin, so it works on any host.
GitHub Repository
Three ways to request Markdown:
- .md suffix : append .md to any post or page URL (e.g. example.com/my-post.md )
- Query parameter : add ?format=markdown to any post or page URL
- Content negotiation : send Accept: text/markdown in the request header
What’s in every response:
- Structured metadata header with title, date, categories, tags, word count, character count, and estimated token count (in YAML frontmatter format, readable by any AI agent)
- Clean Markdown converted from fully-rendered post HTML (shortcodes run, filters applied)
- Content-Type: text/markdown and Vary: Accept response headers
- Content-Signal header for AI signal declaration — defaults to ai-train=no, search=yes, ai-input=yes — see contentsignals.org
- X-Markdown-Tokens header with estimated token count
- Discovery via <link rel="alternate"> in the HTML head and Link HTTP header
- Automatic cache invalidation when a post is updated or deleted
Performance:
Botkibble writes Markdown to disk on the first request, then serves it as a static file. A built-in Fast-Path serves cached files during WordPress’s init hook, before the main database query runs. No extra configuration needed.
Add a web server rewrite rule and Botkibble bypasses PHP entirely, serving .md files the same way a server would serve an image or CSS file:
Method Avg. response time