What Is an llms.txt File and Should Your Website Have One?
What the llms.txt proposal is, what goes in the file, how it differs from robots.txt, and whether it is worth adding.
llms.txt is a small text file that some websites now place at /llms.txt, next to robots.txt. It gives AI tools a short, readable summary of the site and a list of its most useful pages. It is a proposal, not an official standard, and support varies โ but it is cheap to add and easy to understand.
What llms.txt is
The idea, proposed in 2024, is simple: web pages are full of menus, ads and scripts that make them hard for language models to read. An llms.txt file is a clean summary written in Markdown โ a title, a one-paragraph description and links to key pages, each with a short note. An AI tool that reads it gets a quick map of what the site offers.
How it differs from robots.txt and sitemaps
- robots.txt tells crawlers what they may and may not crawl. It is about permission. See what a robots.txt file is.
- XML sitemap lists all your indexable URLs for search engines. It is about completeness.
- llms.txt describes your site and highlights your best pages in plain language. It is about understanding.
llms.txt does not block or allow anything. To control AI crawlers, you still use robots.txt โ read should I block AI crawlers.
What to put in it
- A heading with your site or business name.
- A short summary: who you are and what you offer.
- Sections of links to your most important pages โ services, products, key guides, pricing, contact โ each with a one-line description.
Keep it short and honest. It is a guide to your best content, not a place for keyword lists.
Do AI tools actually use it?
Be realistic. Major search engines have not said they use llms.txt for ranking or answers, and Google has said it does not need it. Some AI tools, developer assistants and documentation platforms do read it. Think of it as low-cost, low-risk: it may help some tools understand your site, and it will not hurt. It is not a substitute for clear, crawlable pages, good structure and accurate information.
How to create one
The free AI crawler checker generates a starter llms.txt from your site name, address, a one-line summary and your key pages. Review it, edit the descriptions in your own words, and upload it to the top folder of your site so it loads at https://yourdomain.com/llms.txt.
If you track AI referrals, you can use UTM-tagged links in the file โ see how to track ChatGPT traffic in GA4.
What a good llms.txt looks like
For a small bakery, a sensible file would start with the business name as a heading, then one paragraph: "Rosie's Bakery bakes custom cakes and eggless desserts in Bristol, with same-day collection and local delivery." Below that, a short "Key pages" list: custom cake orders, eggless cakes, prices and delivery areas, opening hours and contact, each with a one-line description. Optionally, a second list of helpful guides, such as how to choose a cake size.
That is all it needs. Keep it accurate, update it when pages change, and avoid stuffing it with keywords or marketing claims.
Related guides
More free tools: browse all our free AI search tools, no sign-up needed.
Frequently asked questions
Is llms.txt required?
No. It is optional and not an official web standard.
Does llms.txt improve Google rankings?
No. Google does not use it for ranking.
Where should the file go?
In the root of your site, so it is available at /llms.txt.
Can llms.txt block AI bots?
No. Use robots.txt rules to allow or block specific crawlers.