Skip to content

Add built-in llms_txt plugin - #105

Merged
shenxianpeng merged 2 commits into
mainfrom
feature/llms-txt
Sep 24, 2026
Merged

shenxianpeng merged 2 commits into
mainfrom
feature/llms-txt

Conversation

@shenxianpeng

Copy link
Copy Markdown
Member

Adds a built-in llms_txt plugin that publishes the documentation for LLMs and coding agents, following the llms.txt proposal. It is not enabled by default:

site_url: https://example.com/
site_description: Short summary of the project.

plugins:
  - search
  - llms_txt:
      full_output: true   # optional, default false

Output

  • A Markdown version of every page, at the page's source path: docs/guide/install.md → site/guide/install.md. Because the layout mirrors docs/, relative links between pages keep working. For index pages and with use_directory_urls: false, these paths match the llms.txt v2 convention (…/index.md, page.md next to page.html). The content is the page's Markdown after other plugins have processed it (macros and snippets are expanded), without front matter, and starting with the page's title when the page has no H1 of its own.
  • llms.txt at the site root: site_name as the H1, site_description as the summary, and links to the Markdown pages grouped by navigation section (## Guide, ## Guide / More; top-level pages under ## Pages). A page's description metadata becomes the link's note. Links are absolute when site_url is set, relative otherwise.
  • llms-full.txt (with full_output: true): all pages in navigation order in a single file.

Draft pages, external links and anchor links are left out.

Design notes

  • The Markdown versions are made from the pages' Markdown, not from the generated HTML. They're precise, and there are no extra dependencies. The trade-off is that content generated while rendering (e.g. mkdocstrings ::: blocks) appears in its source form. The docs say so.
  • The plugin is named llms_txt so it doesn't clash with the llmstxt entry point of the third-party mkdocs-llmstxt package.

Verification

  • New mkdocs/tests/llms_txt_tests.py covers the full llms.txt output, the per-page Markdown, llms-full.txt, site_url with URL quoting, and path mapping.
  • Built this project's own docs with the plugin enabled, alongside literate-nav, redirects and mkdocstrings: 21 Markdown pages, a 1.8 KB llms.txt, and a 330 KB llms-full.txt, with a --strict build.

Known follow-up: mkdocs get-deps doesn't know llms_txt is built in, because the mkdocs-get-deps package hardcodes search as the only built-in plugin, so it reports llms_txt as unknown.

The release notes rename the upcoming release to 2.0.0.

Related Issue

No issue in this repository.

Checklist

  • New tests added for new behavior (if applicable)
  • Documentation updated (if applicable)
  • Release notes docs/about/release-notes.md updated (if applicable)

🤖 Generated with Claude Code

shenxianpeng and others added 2 commits September 24, 2026 16:09
Publish the documentation for LLMs and coding agents, following the
llms.txt proposal (https://llmstxt.org/):

- A Markdown version of every page, at the page's source path (so
  relative links between pages keep working). It is the page's Markdown
  after plugins have processed it, without front matter, starting with
  the page's title.
- An `llms.txt` index at the site root: `site_name` as the title,
  `site_description` as the summary, and links to the Markdown pages
  grouped by navigation section, with the pages' `description` metadata
  as notes. Links are absolute when `site_url` is set.
- With `full_output: true`, an `llms-full.txt` with all pages in
  navigation order.

Draft pages, external links and anchor links are left out. The plugin is
not enabled by default; enable it with `plugins: [search, llms_txt]`.

The upcoming release is renamed to 2.0.0 in the release notes.

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added the enhancement New feature or request label Sep 24, 2026
@codspeed

codspeed Bot commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 28 untouched benchmarks


Comparing feature/llms-txt (f6c49ff) with main (f85c5e4)

Open in CodSpeed

@shenxianpeng
shenxianpeng merged commit d18aa42 into main Sep 24, 2026
23 checks passed
@shenxianpeng
shenxianpeng deleted the feature/llms-txt branch September 24, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant