If you’ve run a Lighthouse audit in Chrome recently and seen a new “Agentic Browsing” category flagging a missing llms.txt file, this is the explanation. It’s one of the cheapest, fastest pieces of agent-readiness work available — roughly ten minutes of effort — and a majority of major websites still haven’t done it.

What Is llms.txt?

llms.txt is a plain-text, Markdown-formatted file placed at the root of your domain that gives AI agents a curated summary of your website and a short index of its most important pages. Chrome’s Lighthouse documentation describes it as “a machine-readable summary of a website’s content, specifically designed for LLMs and AI agents.”

The reasoning behind it is about token budgets. An AI agent arriving at your site has a limited context window and limited patience. Lighthouse notes that without the file, “agents may spend more time crawling the site to understand its high-level structure and primary content.” An llms.txt file lets an agent read ten high-signal pages instead of guessing its way through a thousand.

How Is llms.txt Different From robots.txt and sitemap.xml?

The three files answer different questions. robots.txt says what bots are allowed to access. sitemap.xml is a flat enumeration of every URL. llms.txt is a human-curated, prioritised summary of what matters and why.

That curation is the whole point. A sitemap treats your privacy policy and your core service page as equals. An llms.txt file says: here is what this business does, here are the six pages that actually explain it, and here’s a one-line description of each. It’s an editorial document, not a generated one — which is also why it can’t simply be auto-exported from your CMS and left alone.

Does Google Use llms.txt for Search Rankings?

No. Google Search has not indicated that llms.txt affects search rankings. Its value is in browser-agent readiness — how cleanly autonomous browsing agents can parse your site — not in conventional search visibility.

This split confuses people, because Google is effectively saying two things at once. Google Search is talking about search visibility, where llms.txt plays no confirmed role. Lighthouse is talking about agent readiness, where it’s one of four audits in the Agentic Browsing category alongside WebMCP integration, agent accessibility, and layout stability. Both statements are true; they’re about different surfaces.

Should Your Business Create One?

Create an llms.txt file if your site has fewer than roughly 1,000 pages and you can maintain the file without burdening your publishing workflow. The maintenance cost is low at small scale, and the file does no harm even if some platforms ignore it.

For most small and mid-sized business websites — service businesses, local businesses, agencies, small ecommerce stores — this describes you comfortably. The calculation gets harder for very large sites with thousands of pages and frequent content changes, where keeping a curated file accurate becomes a genuine ongoing cost.

How to Write an llms.txt File

The format is Markdown, saved as a plain-text file named llms.txt and uploaded to your domain root so it resolves at https://yourdomain.com/llms.txt.

The structure follows a simple convention:

1. An H1 with your site or business name
2. A blockquote with a one-sentence summary of what the business does
3. Optional short paragraphs of essential context
4. H2 sections grouping links by purpose
5. Each link as a Markdown bullet with a one-line description

A Copy-Ready Template

markdown

# Techno Alig

> A web development and digital marketing agency in Aligarh, India, building
> custom coded, WordPress and Shopify websites with SEO and performance built in.

Founded in Aligarh, Uttar Pradesh. Serves clients across India.
Services span website development, ecommerce, SEO, design, and paid advertising.

## Services

- [Website Development](https://technoalig.com/website-development/):
  Custom coded, WordPress and Shopify builds for business websites and web apps.
- [WordPress Development](https://technoalig.com/wordpress/):
  Custom WordPress themes and content-driven builds.
- [Shopify Development](https://technoalig.com/shopify/):
  Ecommerce stores built on Shopify, including migrations.
- [SEO Services](https://technoalig.com/seo-agency-aligarh/):
  Technical SEO, local SEO, and content strategy.
- [Website Design](https://technoalig.com/designing/):
  Responsive UI/UX design for business websites.

## Company

- [About Us](https://technoalig.com/about-us/):
  Team, background, and how the agency works.
- [Previous Work](https://technoalig.com/previous-work/):
  Portfolio of delivered client projects.
- [Contact](https://technoalig.com/contact-us/):
  Enquiry form, phone number, and business address.

## Resources

- [Blog](https://technoalig.com/blog/):
  Articles on web development, SEO, AI tools, and digital marketing.

Adapt the sections to your own business. The principle is restraint: include the pages that genuinely explain what you do and how to engage you, and leave out everything else.

Common Mistakes to Avoid

Dumping your entire sitemap into it. This defeats the purpose. If everything is listed, nothing is prioritised, and you’ve just recreated sitemap.xml in a different format.

Writing marketing copy in the descriptions. Agents are extracting facts, not absorbing persuasion. “Award-winning solutions for your digital journey” tells an agent nothing. “Custom WordPress themes and content-driven builds” tells it exactly what to match against a user’s request.

Letting it go stale. A file that lists services you no longer offer, or links to pages that have moved, actively misleads agents — arguably worse than having no file at all. Review it whenever your service pages change.

Saving it in the wrong place or format. It must be plain text, named exactly llms.txt, at the domain root. A .md extension, a subdirectory, or an HTML wrapper will fail the audit.

How to Add llms.txt to a WordPress Site

Upload the file directly to your site’s root directory via FTP or your hosting file manager — the same directory containing robots.txt and wp-config.php. Then verify it loads at yourdomain.com/llms.txt.

Some hosting environments and security plugins interfere with serving plain-text files from the root. If the file returns a 404 despite being uploaded, check your server’s rewrite rules. Note that Lighthouse marks the audit as “Not Applicable” on a 404, but flags a server error when the file exists yet can’t be retrieved — so a misconfigured file is a worse outcome than no file.

FAQs

Is llms.txt an official web standard? It’s a community convention rather than a formal W3C standard, but it has gained meaningful adoption and is now checked by Chrome’s Lighthouse Agentic Browsing audit, which lends it practical weight.

How often should I update llms.txt? Whenever your core service pages, navigation structure, or business description meaningfully change. For most businesses that’s a few times a year, not monthly.

Does llms.txt stop AI companies from training on my content? No. It’s a discovery and summarisation aid, not an access control or licensing mechanism. Content-use signals belong in robots.txt directives and, increasingly, in ai.txt files.

What’s the difference between llms.txt and WebMCP? llms.txt tells agents who you are and what pages matter — the identity layer. WebMCP lets agents actually perform actions on your site — the capability layer. They solve different problems and a site can need one, both, or neither.

---

llms.txt is the lowest-effort item on any agent-readiness checklist, which is exactly why it’s a strange thing to still be missing from the majority of major websites. Ten minutes of curation buys you a cleaner first impression with every agent that arrives.