llms.txt: The Standard Your Documentation Is Missing

There’s a new file showing up in the wild. You’ll find it at the root of websites, right next to robots.txt and sitemap.xml: llms.txt.

It’s a Markdown file that tells AI agents what your documentation contains and where to find it. Think of it as a table of contents — but written for machines, not humans.

The idea is simple: AI agents need to understand your documentation. But they can’t browse. They can’t click. They can’t hover over a nav bar and figure out the structure. They need a map. llms.txt is that map.

And its existence tells you something important about where documentation is going.

What llms.txt Actually Is

The format was proposed in late 2024 and has since been adopted by a growing number of documentation sites. The specification is straightforward:

A plain Markdown file at /llms.txt that provides:

• A project name and description — what this documentation is about
• Structured links to doc sections — organized by category
• Optional metadata — like update frequency, version, and language

Here’s what a typical llms.txt looks like:

# ProjectName

> A short description of the project.

## Core Documentation

- [Getting Started](https://example.com/docs/getting-started.md): Setup and first steps
- [Authentication](https://example.com/docs/auth.md): API keys, OAuth, and tokens
- [Rate Limiting](https://example.com/docs/rate-limits.md): Limits and retry strategies

## API Reference

- [Users API](https://example.com/docs/api/users.md): CRUD operations for users
- [Payments API](https://example.com/docs/api/payments.md): Processing payments
- [Webhooks](https://example.com/docs/api/webhooks.md): Event-driven integrations

## Guides

- [Error Handling](https://example.com/docs/guides/errors.md): Common errors and how to handle them
- [Pagination](https://example.com/docs/guides/pagination.md): Working with paginated responses

That’s it. No framework. No build step. No AI. A Markdown file that a human writes and a machine reads.

Why It Exists

The problem llms.txt solves is real: AI agents are terrible at navigation.

When a human visits a documentation site, they see the navigation sidebar, the search bar, the breadcrumbs. They understand the structure instinctively. They click around. They find what they need.

An AI agent sees none of this. It gets a URL and a context window. If you point it at your docs homepage, it has to parse the HTML, figure out which links matter, and hope it doesn’t miss anything important. This is slow, imprecise, and expensive in tokens.

llms.txt gives the agent a shortcut. Instead of crawling your site, it reads one file and knows exactly where everything is. It’s faster, cheaper, and more reliable.

This matters because AI agents are consuming documentation at scale. Every time a developer uses Claude Code, Cursor, or Copilot to integrate with your API, the agent needs to understand your documentation. If it can’t find the right page, it guesses. If it guesses, it generates wrong code.

llms.txt reduces guessing.

The Bigger Signal

But here’s what’s interesting about llms.txt: it’s a band-aid. A reasonable, useful band-aid — but a band-aid nonetheless.

The reason llms.txt exists is that documentation was never designed to be machine-readable. It was designed for humans. The navigation, the layout, the progressive disclosure — all of it assumes a human reader who can see the page and make decisions about where to click.

AI agents can’t do that, so we’re adding a separate file to tell them what the page layout tells humans for free.

This is the same pattern we’ve seen across the documentation space. OpenAPI specs exist because code wasn’t designed to describe itself to documentation tools. JSDoc comments exist because function signatures don’t carry enough metadata for humans to understand usage. The entire documentation tooling industry exists because the systems that produce APIs weren’t built to explain them.

Every one of these solutions is a bridge between two systems that weren’t designed to talk to each other.

llms.txt is the latest bridge. It’s useful. It’s also a symptom.

What llms.txt Doesn’t Solve

llms.txt solves the navigation problem. It does not solve the accuracy problem.

An AI agent with a perfect llms.txt can find every page of your documentation. It can’t tell if those pages are correct. It can’t detect that the amount field was changed from integer to string three sprints ago. It can’t flag that the error response schema in the docs doesn’t match what the API actually returns.

The agent reads the documentation, trusts it, and generates code based on what it finds. If the documentation is wrong, the agent generates wrong code. llms.txt just helps it find the wrong documentation faster.

This is the limitation of structural standards. They improve access. They don’t improve accuracy. And in the AI era, accuracy is the thing that matters most.

The Validation Gap Remains

We’ve written extensively about the documentation supply chain — how docs flow through AI agents, MCP servers, IDE integrations, and support workflows. Every one of those consumers needs two things:

1. Access — the ability to find and read your documentation
2. Accuracy — the assurance that what they’re reading is correct

llms.txt solves #1. It does nothing for #2.

For accuracy, you need validation. A system that continuously checks your documentation against your code. Not just at publish time. On every commit. In every environment.

This is the gap that llms.txt doesn’t fill. It’s a navigation file. It’s not a validation layer.

What Good Documentation Infrastructure Looks Like

The teams that get documentation right in the AI era will solve both problems:

Machine-readable structure. Yes, llms.txt helps. So does a well-organized OpenAPI spec. So does consistent Markdown structure. Make it easy for machines to find and parse your docs.

Continuous validation. This is the harder problem. It requires a system that reads your code, reads your docs, and checks that they agree. Not once. Continuously. On every code change.

The first problem has a spec. The second problem requires infrastructure.

At boringdocs, we’re building the second piece. The validation layer that sits between your code and your docs and keeps them in sync. We don’t care whether your docs have an llms.txt. We care whether your docs are telling the truth.

The Standard Is a Start

llms.txt is a good standard. It’s simple, it’s open, and it solves a real problem. If your documentation site doesn’t have one, add it. It takes 30 minutes and helps AI agents understand your docs.

But don’t mistake it for a complete solution. A map to inaccurate documentation is still inaccurate documentation. The agents are getting better at reading. They deserve better content to read.

The teams that understand this will build documentation infrastructure that works for both audiences — the humans who navigate and the agents that execute. Navigation is necessary. Accuracy is non-negotiable.

Start with the map. Then build the validation layer that makes sure the map is honest.

---

Documentation needs machines and humans to both find the truth. Join the waitlist — boringdocs is the validation layer that keeps your docs in sync with your code, continuously. Because AI agents can’t afford to guess, and your docs shouldn’t make them.