SEO Tool

llms.txt Generator

Generate and validate standard llms.txt & llms-full.txt markdown files for AI assistants and developer agents.

Runs in your browser. Nothing is uploaded.
0 links 0 sections
Site & Project Info

Generates the single markdown H1 required by llmstxt.org specification.

Rendered as > summary right under your H1.

Sections & Curated Links

0 lines • 0 chars • 0 bytes (UTF-8 LF)
Where to deploy your file

Upload this file to the root directory of your website so it is served at: https://yourdomain.com/llms.txt with Content-Type text/markdown; charset=utf-8 or text/plain.

Optional robots.txt recommendation:
User-agent: *
Allow: /llms.txt
Allow: /llms-full.txt

How to Use the llms.txt Generator

  1. Define your project basics: Enter your website or product name as the primary H1, along with a concise one-sentence description in the summary blockquote.
  2. Curate sections and links: Organize key documentation, API endpoints, tutorials, and architecture guides into logical H2 sections. You can mark lower-priority links under an Optional section.
  3. Or import your existing sitemap: Switch to the Import tab to paste your XML sitemap or URL list. The tool automatically groups paths, converts slugs to clean titles, and allows pruning.
  4. Review instant validation: Check the Validation tab for missing descriptions, non-HTTPS protocols, broken URL structures, or domain mismatches.
  5. Export & deploy: Copy or download your clean llms.txt file and place it at the root of your web server (e.g. https://example.com/llms.txt).

llms.txt Example: Before and After

Unlike verbose HTML sitemaps cluttered with navigation chrome, scripts, and promotional banners, llms.txt provides a deterministic, lightweight index in standard markdown.

Standard sitemap.xml snippet
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://fastdocs.example.com/docs/quickstart</loc>
    <lastmod>2025-02-15</lastmod>
  </url>
  <url>
    <loc>https://fastdocs.example.com/api/rest</loc>
  </url>
</urlset>
Equivalent clean llms.txt output
# FastDocs

> High-speed vector search and documentation for AI agents.

## Core Documentation
- [Quickstart Guide](https://fastdocs.example.com/docs/quickstart): 5-minute setup and verification
- [REST API Reference](https://fastdocs.example.com/api/rest): Endpoints, schemas, and rate limits

## Optional
- [Changelog](https://fastdocs.example.com/changelog): Release notes and deprecations

Explanation: What is llms.txt and Why Does It Matter?

The llms.txt proposal, introduced by Answer.AI and widely adopted across the open-source and developer tools community, creates a standard discovery point for Large Language Models (LLMs) and context retrieval engines. When AI systems like Cursor, Claude Projects, ChatGPT, or autonomous code agents attempt to assist developers, they often struggle when scraping bloated HTML pages filled with navigation bars, cookies banners, and client-side JavaScript hydration.

By hosting a curated /llms.txt file at your domain's root, you present an opinionated directory of clean, machine-friendly markdown resources. This drastically cuts down token consumption, eliminates scraping hallucination, and ensures that agents read the exact canonical references you intend them to digest.

Common Problems & Limitations

Honest note on crawler support: llms.txt is a community-proposed convention rather than a W3C standard or an RFC. While developer tools and AI models widely embrace it, search engine webmasters and AI scrapers are not legally or architecturally bound to honor it.
  • Bloated link lists: Pasting an entire 10,000-page e-commerce sitemap into llms.txt defeats its core purpose. The file should be curated to the top 50–100 high-signal documentation files.
  • Relative URLs: AI agents reading llms.txt in isolation without host context cannot reliably resolve relative paths like /docs/intro. All links must be fully qualified HTTPS URLs.
  • Missing summaries: While brief link lists work, giving each link a concise 5–15 word description provides vital semantic context for retrieval-augmented generation (RAG) routers.

Frequently Asked Questions

What is an llms.txt file?

An llms.txt file is a standardized markdown file placed in your website root (like robots.txt or sitemap.xml) designed specifically to provide curated, high-information-density documentation links for LLMs and AI agents.

Do AI crawlers and LLMs actually read llms.txt?

Support is not guaranteed across all AI vendors. However, modern developer environments like Cursor, Claude Code, Windsurf, Perplexity, and custom RAG pipelines increasingly prioritize checking /llms.txt to fetch ground-truth markdown documentation without HTML noise.

Where should I upload my generated file?

Upload the file to your web server's public root so that it resolves at https://yourdomain.com/llms.txt. Ensure your HTTP server or CDN delivers it with a plain text or markdown MIME type (e.g., text/plain or text/markdown with UTF-8 encoding).

What is the difference between llms.txt and robots.txt?

robots.txt tells web crawlers where they are permitted or forbidden to go. llms.txt is content-focused: it directs AI models to the highest quality, noise-free markdown content for your product or documentation.

What is llms-full.txt?

While llms.txt is a curated index of links, llms-full.txt concatenates the full text of all your key documentation pages into a single large markdown file, allowing an LLM with a 200k+ token context window to ingest your entire library in one prompt.

Does user data ever leave my browser?

No. MafLabs tools run entirely client-side. Your sitemaps, draft URLs, and configuration are processed purely in local browser memory and never sent to any server or remote API.