llms.txt Generator
Turn your sitemap or URL list into a spec-compliant llms.txt file that helps AI assistants understand your site. Everything runs in your browser — nothing is uploaded.
Follows the llmstxt.org proposed standard
Tip: if you paste a sitemap index (a sitemap of sitemaps), open each child sitemap and paste its contents here instead. Fragments (#section) are stripped and duplicates removed automatically.
About llms.txt
What is llms.txt?
llms.txt is a proposed standard, introduced by Jeremy Howard of Answer.AI in September 2024, for giving large language models a curated, markdown-formatted map of your website. Where a human visitor gets navigation menus and search, an AI assistant fetching your pages gets raw HTML full of scripts, banners, and boilerplate. An llms.txt file placed at the root of your domain cuts through that: a single plain-text page listing your most important URLs, each with a one-line description, organized under headed sections.
The format is deliberately minimal. It starts with an # H1 naming the site (the only required element), an optional > blockquote summary, optional free-form context, and then ## sections of markdown link lists in the form - [Title](url): description. A special section named ## Optional marks links that can be skipped when an AI is working with a tight context window.
The honest state of adoption
It is worth being clear-eyed here: llms.txt is a community proposal, not an official protocol. As of 2026, crawl studies put adoption somewhere around 9–16% of top websites — growing steadily, and notably strong among developer documentation sites (Anthropic, Stripe, Cloudflare, and many others publish one). However, no major LLM provider has officially committed to fetching llms.txt during training or browsing, and Google's search team has publicly downplayed it. So why bother? Because the cost-benefit is lopsided: it takes minutes to generate, costs nothing to host, cannot hurt your SEO, and if AI assistants do consult it — some agentic browsing tools already do — your site is represented by the pages and descriptions you chose rather than whatever a crawler happened to parse. Think of it as cheap insurance on AI discoverability.
llms.txt vs robots.txt vs sitemap.xml
| File | Audience | Purpose | Format |
|---|---|---|---|
| robots.txt | Crawlers | Sets rules — which paths bots may or may not crawl | Directive syntax (User-agent, Disallow) |
| sitemap.xml | Search engines | Exhaustively lists every indexable URL for discovery | XML, machine-only |
| llms.txt | LLMs and AI agents | Curates and explains your best content with context | Markdown, readable by humans and machines |
The three are complementary, not competing. robots.txt says "here is what you may not touch," sitemap.xml says "here is everything that exists," and llms.txt says "here is what actually matters, and why." That is also why this generator asks you to prune: pasting your entire sitemap and including all of it defeats the purpose. A good llms.txt is a highlight reel, not an index.
How to deploy your llms.txt
- Location: serve it at
https://yourdomain.com/llms.txt— the root of the domain, next to robots.txt. - Content type: it should be delivered as
text/plain; most static hosts do this automatically for .txt files. - Static sites: drop the file into your
public/orstatic/directory (Nuxt, Next.js, Astro, Hugo all support this). - Keep it fresh: update it when you launch major pages — a stale llms.txt that 404s half its links is worse than none. This tool saves your work locally so you can come back and regenerate.
Writing one-liners an AI can quote
The description after each link is the highest-leverage text in the file. Write it as a plain factual sentence fragment that could stand alone in an AI's answer: say what the page contains and who it is for, not marketing copy. "Complete REST API reference with authentication examples" beats "Check out our awesome docs!" every time. Keep descriptions to one line, lead with concrete nouns, and avoid duplicating the title — the description should add information, not restate it.
Frequently Asked Questions
Do AI companies actually read llms.txt files?
Honestly: no major LLM provider has officially committed to it yet. OpenAI, Anthropic, and Google do not document llms.txt as part of their crawling pipelines, and Google Search has said it does not use the file. That said, many AI documentation tools and agentic browsers do fetch it, thousands of developer-focused sites publish one, and adoption keeps growing. It costs minutes to create and nothing to host, so most site owners treat it as cheap insurance.
Where do I put the llms.txt file on my site?
Place it at the root of your domain so it is reachable at https://yourdomain.com/llms.txt, the same location convention as robots.txt. It should be served as plain text (text/plain), which virtually all hosts do automatically for files ending in .txt. On static frameworks like Nuxt, Next.js, or Hugo, just drop the file in your public or static folder.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a curated index: links plus one-line descriptions that an AI can follow selectively. llms-full.txt is a companion convention where sites concatenate the full text of their documentation into one large markdown file, so an LLM can ingest everything in a single request. Start with llms.txt; llms-full.txt mainly makes sense for documentation-heavy sites.
Does llms.txt block AI crawlers like robots.txt does?
No — it does the opposite. robots.txt sets access rules that tell crawlers what they may not fetch. llms.txt is an invitation: it curates the pages you want AI systems to read and explains what each one contains. If you want to restrict AI training crawlers, that still happens in robots.txt with user-agent rules; the two files work side by side.
Is my sitemap or URL list uploaded to a server?
No. This generator runs entirely in your browser: sitemap parsing uses the built-in DOMParser, the llms.txt is assembled in JavaScript on your device, and your work-in-progress is saved only to your browser's localStorage. Nothing you paste ever leaves your machine unless you explicitly click Share.
Related Tools
Working on your site's technical files? Try our JSON Formatter for structured data, Base64 Encoder for data URIs, or Text Case Converter for cleaning up titles.
Discussion
Start the conversation
Leave a comment
Loading comments...