What is llms.txt, and does it help your brand appear in AI answers?

llms.txt is a proposed Markdown file, usually placed at a site's root (/llms.txt), that gives AI agents a short summary of the site and links to its most useful pages. It is a voluntary convention: Google says its Search ignores it, and OpenAI, Anthropic and Perplexity do not mention it in their crawler documentation.

The idea is simple. Web pages are built for people, with menus, scripts and adverts around the content. An AI agent working for a user has limited room in its context window, so a curated map of the site, in plain Markdown, saves it from guessing. Whether that map changes which brands AI assistants recommend is a different question, and so far nobody has documented that it does.

In short

• Jeremy Howard proposed llms.txt on 3 September 2024. A second version followed in August 2026.
• The file is Markdown: a title, a one-paragraph summary, then lists of links with short notes.
• OpenAI, Anthropic and Google publish llms.txt files for their own developer documentation, and the proposal says the files are used most heavily by coding agents reading software documentation.
• Google states that Google Search does not use llms.txt and that having one will neither help nor harm your visibility in Google Search, including AI Overviews.
• It is cheap to publish and harmless, but do it after the basics: readable HTML and a deliberate robots.txt.

Who proposed llms.txt, and why?

Jeremy Howard published the proposal at llmstxt.org on 3 September 2024. Its GitHub repository sits under Answer.AI, and the proposal says it is "open for community input". It is not a standard from a standards body such as the IETF or W3C.

The proposal's argument is that HTML is hard for language models to use: converting a page full of navigation and JavaScript back into clean text "is difficult and imprecise", and most websites are too large to fit in a model's context in full. The second version, published in August 2026, adds standard link relations so an agent can find a page's Markdown version and the llms.txt that covers it, and it lets a file at a subpath such as /docs/llms.txt cover just that section.

The proposal is also clear about its purpose. Its author expected llms.txt "would mainly be useful for inference rather than training", meaning it is read on demand while an agent helps a user, not fed into a model during training.

What does an llms.txt file look like?

The format, in the order the proposal specifies:

1. An H1 with the name of the site or project. This is the only required part.
2. A blockquote with a short summary containing the key information needed to understand the rest.
3. Optional detail: paragraphs or lists, but no headings.
4. Sections under H2 headings, each a list of links in the form [name](url), optionally followed by a colon and a note.
5. An "Optional" section, by convention, for secondary links an agent can skip when it is short of space.

For example, a fictional South African bank might publish:

# Meridian Bank

> Meridian Bank is a South African retail and business bank. This file lists our product pages, fees and support guides.

Fees are quoted in rand and include VAT.

## Products

- [Business current account](https://www.example.com/business/current-account.md): fees, limits and how to apply
- [Savings accounts](https://www.example.com/savings.md): interest rates and notice periods

## Optional

- [Media releases](https://www.example.com/media.md)

The proposal also suggests offering a clean Markdown copy of important pages at the same address with .md added, such as page.html.md, and pointing llms.txt at those copies.

Who uses llms.txt today?

What we could verify on 11 October 2026:

Who	What they do with llms.txt
OpenAI, Anthropic, Google	Publish llms.txt files for their developer documentation (developers.openai.com, docs.anthropic.com, ai.google.dev)
Documentation platforms and CMSs	The proposal lists Mintlify, GitBook, Yoast SEO, AIOSEO and Wix among tools that generate the file automatically
Chrome Lighthouse	Includes an llms.txt audit that flags a server error when fetching the file; a missing file is marked not applicable
Google Search	Ignores it. Google says Search "doesn't use" llms.txt or similar files

Two things stand out. The AI labs publish llms.txt to help agents read their own documentation; OpenAI's, Anthropic's and Perplexity's crawler pages do not say their crawlers read anyone else's. And Lighthouse's documentation calls llms.txt "an emerging convention" and says providing one is "optional at the moment".

Does llms.txt help you appear in AI answers?

There is no published evidence that it does.

Google is explicit. Its guide to generative AI features in Search, updated in July 2026, lists llms.txt under things you can ignore. You "don't need to create new machine readable files, AI text files, markup, or Markdown" to appear in Google Search, "including its generative AI capabilities", and maintaining an llms.txt file "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them". Google added that clarification on 15 June 2026.

OpenAI, Anthropic and Perplexity are silent. Their crawler documentation explains robots.txt controls in detail and does not mention llms.txt as something their crawlers read. Silence is not proof they never look at it, but it means no one can promise you a benefit.

Where it does seem to help is the use the proposal describes: agents that look for the file while doing a task, especially coding agents reading software documentation. If developers integrate with your product, an llms.txt for your API docs is a sensible courtesy.

For a typical South African marketing site, the things that demonstrably matter come first: words in the HTML that crawlers receive (see is your website readable by ChatGPT?) and a robots.txt that does not block the search crawlers you want (see robots.txt for AI crawlers).

How is llms.txt different from robots.txt and sitemap.xml?

File	Purpose	Who reads it
robots.txt	Says which parts of the site crawlers may access	Search and AI crawlers, before crawling
sitemap.xml	Lists every page you want indexed	Search engines
llms.txt	A curated summary and reading list in Markdown	AI agents that choose to look for it, on demand

The proposal frames llms.txt as a complement to both: a sitemap lists everything, while llms.txt picks what an agent needs and can link to pages on other sites. Schema markup is a separate layer again; see structured data for AI search.

Should a South African business publish one?

If it takes an hour, yes, as long as nobody sells it to you as a ranking fix. CitedAI publishes one at citedai.co.za/llms.txt, generated from the blog each time the site is built, so it never goes stale. We do not count on it to be cited.

If you write one, the proposal's own guidelines are good ones:

• Use concise, clear language.
• Give every link a brief, informative description.
• Avoid ambiguous terms and unexplained jargon.
• Test it by asking an AI agent questions about your business with only the llms.txt as a starting point.

Keep the facts in it, such as your name, what you sell and where you operate, identical to your website and profiles. Consistency helps everywhere; a file that contradicts your site helps nowhere.