Generate llms.txt from your website
Enter a domain. The generator finds its sitemap, reads each page’s title and description, and gives you an editable draft.
Checks robots.txt and XML sitemaps, then reads up to 100 public HTML pages. A scan can take several seconds.
Review before you publish
The sitemap gives you coverage, while the editor gives you control. Remove duplicate, account, search, tag, and other low-value pages. Keep the routes that explain the product, answer important questions, or help someone complete a task.
- Review the scan. Edit titles and descriptions, choose a section, and exclude pages that add little context.
- Download the file. Keep the filename exactly
llms.txt. - Publish it at the website root. The public URL should be
https://example.com/llms.txt. - Maintain it with the site. Regenerate it when important pages or canonical URLs change.
Frequently asked questions
How does the generator find pages?
It checks sitemap declarations in robots.txt, falls back to /sitemap.xml, follows sitemap indexes, and reads metadata from each public HTML page.
What happens when a site has more than 100 pages?
The scan stops at 100 pages to keep the tool responsive. That is enough for a curated llms.txt draft; large sites should keep only their most authoritative pages.
Does llms.txt control AI crawlers?
No. Use robots.txt and access controls for crawler permissions. llms.txt is a discovery and context file.