What Is llms.txt, and Does Your Site Need One?
llms.txt is a short Markdown file pointing AI tools to your key pages. It blocks nothing and Google Search ignores it, so for most creators it is optional.
By Alex Nazarov
On this page
llms.txt is a short text file at the root of a website that lists its most important pages for AI tools. It does not block or permit anything, and Google says its search ignores it. For most creators it is optional: useful as a tidy menu, never a replacement for good pages.
You may have heard that llms.txt tells AI how it is allowed to use your work. It does no such thing. Think of it as a table of contents you leave out for any visitor who wants one. iSonic publishes one, and we will walk through ours below.
The short version
| File | What it does | Who says they read it | Does it control AI use? |
|---|---|---|---|
| llms.txt | Offers AI tools a short, curated menu of your key pages | No major AI company documents reading yours. Google Search ignores it | No |
| robots.txt | Tells crawlers which parts of your site they may access | OpenAI, Anthropic and Perplexity all document it | Yes, for crawling |
| sitemap.xml | Lists every page on your site for search engines | Search engines | No |
| Structured data | Labels on each page describing what is on it | No |
Only robots.txt changes what a crawler may do.
What is an llms.txt file?
It is a plain text file written in Markdown. The file sits at yoursite.com/llms.txt and gives AI tools a short summary of your site. It also links to your key pages. Markdown uses ordinary characters to add headings and links to text. The root is the top level of your site, so the file sits alongside your home page and not inside a folder.
The shape is deliberately plain. It includes a title and a one-line summary of your site. Links appear in short lists under headings. Each link has a brief description.
Jeremy Howard proposed the format on llmstxt.org on September 3, 2024. The page currently describes the proposal as version 2. Its author says it was updated after two years of adoption. The convention is still changing.
Is llms.txt the file that tells AI how it can use my work?
No. This is a common misunderstanding about it. If you believe it, you may think your work is protected when it is not.
The proposal explains that robots.txt and llms.txt serve different purposes. Robots.txt tells automated tools which parts of a site they may access. These tools include search engine bots. Llms.txt gives AI agents information to use when helping a user with a topic.
In other words, a tool may fetch the file when it wants context. No one is required to read it. Reading it does not grant permission to use your content in any particular way.
The major AI companies document how their crawlers follow robots.txt, including OpenAI, Anthropic and Perplexity. Check this file if your concern is access.
Do ChatGPT, Claude or Perplexity read it?
Not according to any of them. We checked each company’s crawler documentation on October 18, 2026. None of those documents says its crawler reads a site’s llms.txt file.
Each company does explain how it uses robots.txt. Anthropic says its bots follow “do not crawl” instructions in robots.txt. Perplexity says site owners can add instructions to robots.txt to manage how Perplexity interacts with their sites and content. OpenAI documents robots.txt controls on its own bots page, which our step-by-step guide covers.
These same companies do publish llms.txt files, for their own developer documentation. The llmstxt.org proposal lists OpenAI, Anthropic and Gemini as examples. Perplexity’s crawler page also points agents to docs.perplexity.ai/llms.txt. This file lists links to its documentation.
So these companies use the file for their own docs. None of them says its crawlers read yours.
What does Google say about llms.txt?
Google is the most direct source on this. Google’s AI optimization guide, last updated July 10, 2026, makes this clear. You do not need special AI text files, markup or Markdown to appear in Google Search. This also applies to its AI features. Google Search does not use these files.
The same page says these files do not help or hurt your site’s visibility or rankings in Google Search. Google Search ignores them.
Google added this clarification on June 15, 2026. The changelog entry is titled “Clarifying guidance on llms.txt files”. It says you can keep the file for other services or systems that use it.
This applies to Google Search, including AI Overviews and AI Mode. It does not apply to every Google product.
Google Search also does not penalize the file, so adding one is low risk.
Chrome’s Lighthouse tool checks for the file during its agentic browsing audits. That is why it appears in site reports. Lighthouse describes it as an emerging way to give machines a summary of a website’s content. The file is currently optional, so Lighthouse marks a missing file as “not applicable.”
What does a real llms.txt look like?
It is shorter and plainer than most people expect. Here is the top of ours, at isonic.ai/llms.txt.
# iSonic
> Creator knowledge formatted so people and AI can find, cite, and recommend it.That is the opening. It has a title, then one line saying what the site is. The angle bracket marks that line as a summary in Markdown. Our file also includes a few notes about how our web addresses are structured. These notes are optional and specific to our site.
Below it are some sections of links, each with a short note. Two lines from the Product section:
## Product
- [AI Bio Page](https://isonic.ai/features/ai-bio-page/): One structured page that pulls your content and links together so AI assistants can understand and credit you.
- [GEO Questions](https://isonic.ai/features/geo-questions/): Question-and-answer pairs generated from your content that mirror how people actually prompt AI.Each line is a link, then a sentence saying what sits behind it. That sentence is the only real difference between this and a sitemap. A sitemap lists addresses. This says what each address is for, in words a reader or a tool can use to decide whether to bother.
The whole file runs 63 lines, with 8 headings and 24 links across its sections. That is on the long side for what this is. Most sites would need half of it.
One heading is worth knowing about:
## OptionalThe proposal reserves that section for links an AI tool can skip when it is short on context. Ours holds robots.txt, the RSS feed and the legal pages. They are useful to list, and nothing is lost if a tool skips them.
Should a creator add one?
It depends on one thing: whether you have twenty spare minutes and a site you can upload files to.
If you have your own site and twenty spare minutes, add it. It cannot hurt you in Google, it is tidy, and if the convention gets more traction you are already there.
If you are short on time, skip it with a clear conscience. Focus on pages that answer people’s real questions. How Does ChatGPT Choose Sources? walks through what every engine reads.
If you have no website, this one is not for you at all. An llms.txt file is a menu pointing at pages, so it only means something once pages exist.
If you have a website, check whether you already have one. Open yoursite.com/llms.txt in a browser. Some site builders add the file for you.
How do you make one?
You do not need a developer.
- Write the file in any text editor, or use an llms.txt generator.
- Upload it to the root of your site, so it sits next to your home page.
- Check that it opens at yoursite.com/llms.txt in a browser.
Some site builders do not let you put files at the root. If yours does not, that is a fine place to stop. You are not missing anything an engine was going to read.
What can nobody promise?
No file gets you cited, and neither does any tool, iSonic included.
AI assistants decide for themselves whether to search, what to read and who to credit. No AI company has published anything saying an llms.txt file plays a part in any of those three steps.
Where does iSonic fit?
iSonic is a GEO platform that turns a creator's existing content into structured, AI-readable pages so AI assistants can find it and credit it to the creator. AI assistants can read these pages. This helps them find your content and credit you. Your AI Bio Page explains who you are. GEO Questions draw on content you have already published, in semantic HTML with Schema.org markup. iSonic.ai also publishes one llms.txt file for the whole site. New questions are added as you publish new content.
Quick answers
Is llms.txt mandatory?
No. It is a proposal, not a standard, and no AI company requires it. Google says its search ignores the file. Keeping one will not help or hurt your rankings. The file is optional. If it is missing, Chrome’s Lighthouse audit marks it as “not applicable.”
How do I submit an llms.txt file?
There is nothing to submit. The file only has to be available at yoursite.com/llms.txt. None of the crawler documents we checked mentions a submission step. Google Search also ignores the file. There is no form to fill in, dashboard to use or verification to complete.
Is llms.txt actually used?
More site owners are using it. Its author says the current version reflects two years of adoption. Major AI labs also publish these files for their own documentation. There is no published evidence that their crawlers read the file on your site.
What is llms-full.txt?
A separate convention, mainly used by documentation sites. Mintlify supports this file format. It says the file brings all your site’s documentation into one place. AI tools can use it as context, and LLMs can index it. The llms.txt proposal itself does not mention it.
A file that lists your pages is only as useful as the pages it lists.