Google’s llms.txt Guidance Depends On Which Product You Ask

Google’s Search and Chrome documentation now level in several instructions on llms.txt, relying on whether or not the objective is Search visibility or agentic browser readiness.

Google Search just lately revealed a brand new optimization guide that lists llms.txt among the many techniques you don’t want for generative AI options. The information teams it with content material chunking, AI-specific rewriting, and particular schema.

Days earlier, Google’s Lighthouse instrument shipped model 13.3, which added a brand new Agentic Browsing category. The replace contains an llms.txt audit that checks whether or not a web site gives the file and flags server errors when retrieving it.

The Lighthouse documentation describes llms.txt as a method to supply “a machine-readable abstract of an internet site’s content material, particularly designed for LLMs and AI brokers.” It provides that with out the file, “brokers might spend extra time crawling the positioning to grasp its high-level construction and first content material.”

What Google Search Has Stated

Google’s Search staff has maintained for over a yr that llms.txt will not be a Google initiative or one thing Google plans to undertake.

John Mueller compared llms.txt to the keywords meta tag, noting no AI providers used it and bots didn’t request the file. He known as constructing separate Markdown pages for bots “a stupid idea.”

At Search Central Live Deep Dive Asia Pacific, Gary Illyes and Amir Taboul confirmed Google was not pursuing llms.txt.

Google’s optimization guide explicitly states llms.txt must be skipped, offering the latest direct assertion from the Search staff.

What Chrome’s Lighthouse Now Does

Lighthouse 13.3 ships with the Agentic Shopping class by default, checking WebMCP integration, agent accessibility, structure stability, and llms.txt.

The llms.txt audit solely marks websites as “Not Relevant” in the event that they return a 404; errors flag the audit. The Lighthouse docs describe llms.txt as an “rising conference” at llmstxt.org, advising web site homeowners to create and place it of their root listing.

This class is separate from website positioning audits and signifies that llms.txt helps browser-based brokers perceive web site construction, not enhance search rankings or AI citations.

Google Has Been Right here Earlier than

Google’s inner groups have despatched combined alerts on llms.txt earlier than.

In December, Lidia Infante spotted an llms.txt file on Google’s Search Central developer documentation. Mueller responded on Bluesky with “hmmn :-/” and didn’t make clear additional.

Dave Sensible noted that the file appeared on a number of Google developer properties, together with developer.chrome.com and internet.dev. The sample advised an inner CMS platform replace that mechanically deploys llms.txt information, not a Search staff choice.

The Search Central file was eliminated inside hours, however information on different Google properties remained.

Why This Issues

Google’s reply on llms.txt varies by use case.

For Google Search, llms.txt isn’t wanted for AI Overviews, AI Mode, or different generative AI Search options.

For browser-based brokers, Lighthouse considers llms.txt elective in an experimental machine interplay class.

Steering is cut up between totally different Google developer websites, which might result in conflicting directions when evaluating Lighthouse or its llms.txt documentation with Google’s Search docs.

Wanting Forward

Google hasn’t commented on the documentation hole between the 2 product groups.

For a lot of websites, making a primary llms.txt file is straightforward, however sustaining it’s questionable, on condition that Google Search states it’s pointless for AI Search visibility.

Featured Picture: Inventory-Asso/Shutterstock