# Agent-readable documentation

> How Supacharger Docs publishes XML discovery, llms.txt, llms-full.txt, and raw Markdown pages for search engines and AI agents.

# Agent-readable documentation

Supacharger Docs publishes complementary discovery formats for traditional search crawlers and AI agents. These artifacts are generated from the same Markdown sources as the rendered documentation so they remain aligned during every production build.

## Public endpoints

| Endpoint | Purpose |
| --- | --- |
| [`/robots.txt`](https://supacharger.dev/robots.txt) | Declares crawler access and the absolute XML sitemap location. |
| [`/sitemap.xml`](https://supacharger.dev/sitemap.xml) | Exhaustive standard list of public rendered routes. |
| [`/llms.txt`](https://supacharger.dev/llms.txt) | Concise Markdown index with descriptions and links to raw documentation pages. |
| [`/llms-full.txt`](https://supacharger.dev/llms-full.txt) | Complete public documentation combined into one Markdown document. |
| `/docs/<route>.md` | Raw Markdown alternative for an individual rendered documentation route. |

For example, the hosted setup wizard is available as rendered HTML at `/docs/guides/setup-wizard` and as Markdown at `/docs/guides/setup-wizard.md`.

## Discovery links

Every rendered page advertises `/llms.txt` with:

```html
<link rel="describedby" type="text/markdown" href="/llms.txt">
```

Every rendered documentation page also advertises its matching Markdown resource:

```html
<link
  rel="alternate"
  type="text/markdown"
  href="/docs/guides/setup-wizard.md"
>
```

The conventional root filename and these link relations let an agent discover the machine-oriented representation without guessing or scraping the rendered navigation.

## Build behavior

The Docusaurus sitemap plugin generates `sitemap.xml`. `docusaurus-plugin-llms` generates `llms.txt`, `llms-full.txt`, and individual Markdown pages. A small local Docusaurus plugin generates `robots.txt` after the build so its policy cannot drift away from `noIndex`.

Do not edit files under `build/`; they are disposable output. Edit the source document, its front matter, or the generator configuration and rebuild.

Descriptions in `llms.txt` come from each page's `description` front matter when present. Public pages should therefore have a concise, factual description that distinguishes the page from nearby documentation.

## Public and private builds

Production builds are indexable by default. Development uses Docusaurus `noIndex`, and a private preview or staging deployment must set:

```bash
SITE_NO_INDEX=true
```

That setting prevents sitemap generation, emits robots rules that disallow crawling, and adds Docusaurus no-index metadata.

<!-- TEMPORARY DEVELOPMENT ACCESS GATE — TODO: remove this paragraph when the gate is removed. -->
During the temporary private-review period, `HIDE_DOCS=true` builds the content behind a browser-side development gate instead of omitting it. A valid `token` query parameter unlocks the site for the current tab and is propagated to internal links. The gated build remains `noIndex` and omits search and machine-readable documentation bundles. This is a review convenience rather than secure authentication because the token is present in the static client bundle. Restore build-time omission when the review period ends.

## Maintenance checks

After adding, moving, or deleting documentation, run a production build and verify:

```bash
pnpm run build
```

Then confirm:

- `build/robots.txt` allows production crawling and points to the canonical sitemap;
- `build/sitemap.xml` contains the rendered public route;
- `build/llms.txt` links to the generated Markdown route;
- the individual `.md` file exists under `build/`;
- `build/llms-full.txt` contains the new content; and
- the rendered HTML includes both `describedby` and Markdown `alternate` links.

`llms.txt` complements, rather than replaces, robots policy, XML sitemaps, canonical URLs, structured data, accurate content, and ordinary search-engine indexing.