Google Says Markdown For AI SEO Strips Away The Parts That Matter

On a latest Search Off the File podcast, hosts John Mueller and Martin Splitt pushed again on the concept promoted by AI SEOs that stripped-down, content-only variations are a greater approach to optimize for AI Search. They made the case that each one the issues AI SEOs wish to take away are literally helpful for rating.

Non-Content material Elements Of Net Pages Matter

The TL;DR of this half is that HTML is for browsers to render into a visual web page for people, in addition to for display readers to learn.

Martin Splitt begins the dialogue by explaining why plain HTML seems to not be the best method to offer content material to AI brokers and LLMs. The thought is that, along with content material, there’s plenty of different code within the HTML that’s irrelevant for an LLM or AI agent which may be visiting a website for the content material.

The enchantment of markdown, then, is that it might present the content material in a fashion that breaks freed from all of the HTML that’s meant to make an online web page seen for people or readable by a display reader.

Splitt explains:

“And I believe that’s additionally why individuals assume it’s good for LLMs, as a result of you might have much less stuff, much less tokens. And when you take a look at an HTML file with out a browser rendering it, when you simply take a look at the plain HTML in a textual content editor, principally, then it’s laborious to learn the content material, as a result of there’s a lot cruft, a lot stuff in it. There’s all these HTML tags and all this perhaps even inline types and all that sort of stuff.”

He additionally praises markdown for the flexibility to nonetheless talk the essence of the content material:

“But when a Markdown render fails and also you take a look at the Markdown file in a textual content editor, it nonetheless is structured and readable. Like a hyperlink is the phrase of the hyperlink textual content, just like the anchor textual content, after which in sq. brackets after which in regular brackets. It’s in all probability what I might do if textual content was all I had accessible.

If I used to be writing an e mail with out the chance to really hyperlink issues, I might in all probability mark up some form of hyperlink textual content after which put some form of approach to say, like, and that is the place you should go to really see that.

And I believe this minimalism might be what makes individuals assume, yeah, that is nice for a machine that should perceive this content material, in contrast to HTML.”

Changing HTML To Textual content Is Trivial

Mueller and Splitt famous that regardless of how advanced HTML seems to be, crawling and making sense of it’s trivial and really straightforward to do. The promoting level about utilizing markdown for LLMs, that it simplifies crawling and indexing content material, utterly breaks down at this level.

John Mueller explains:

“I believe the large factor is that the net with HTML and every little thing has been round for actually very long time, longer than Markdown. And all the crawlers on the market, have practiced with HTML. And changing HTML into textual content is trivial. There are many libraries on the market that may try this for you. So if you consider what a mean net crawler may search for or may want to search out on a web page to have the ability to perceive it, then in all probability that’s simply HTML.”

Markdown Fails For Content material Discovery

Discovery is when any crawler visits an online web page and discovers different net pages inside a single web site, and likewise from web site to web site.

Splitt mentioned that markdown is targeted on only one a part of the content material: the content material itself. He defined that this makes it more durable for search engines like google and yahoo to see an online web page within the context of the way it connects to the remainder of a web site’s content material by way of hyperlinks, which assist discovery.

He defined:

“Yeah, and I imply, the opposite factor is, sure, it’s good that Markdown is normally then specializing in a chunk of content material, however HTML with all of the hyperlinks and navigation and the headers and all that sort of stuff that sort of will get stripped out within the Markdown recordsdata that make the web site are essential to grasp the construction and the way this connects to the remainder of the positioning.

So I assume that’s additionally a nasty factor. If we have been to lose this, that’s in all probability not so good for crawling in Discovery, huh? “

Takeaway

Studying patents and analysis papers, it turns into clear that search engines like google and yahoo see a web site as a set of particular person net pages, but additionally as teams of net pages that belong to sections and classes, and likewise as all the web site itself as an entire. Zoom out, and the web site is however one level amongst 1000’s and 1000’s of different web sites in a neighborhood of internet sites, self-organized by hyperlinks into classes and high quality ranges.

For website positioning, we have now to grasp a website from each the zoomed-out and zoomed-in view to conceptualize how all of the items match collectively. The reason being as a result of that’s what search engines like google and yahoo do.

AI-based website positioning appears to be hung up on making it straightforward for LLMs and AI brokers to crawl and index content material. Crawling and indexing are legitimate issues. However by insisting on markdown recordsdata, they aren’t contemplating the basics of discovery and the way trivial it’s to extract content material from an HTML net web page, which makes markdown recordsdata redundant.

Other than the above points, there may be additionally the one about trustworthiness. There was a factor referred to as a key phrase meta tag that some search engines like google and yahoo used to get a touch about what an online web page was about. Naturally, website house owners and SEOs used it to dump all of the key phrases they needed to rank for, whatever the content material.

I’m not saying that SEOs and web site house owners are untrustworthy, however search visitors is cash, and persons are going to do what they’re going to do. So the final consideration is that search engines like google and yahoo won’t ever belief markdown content material and use it because the canonical when it’s a trivial factor to crawl and extract the unique content material from the HTML.

Circling again to what Mueller and Splitt mentioned, Google insists that the AI website positioning insistence on markdown strips away a big quantity of context that issues.

Watch Search Off The File Episode 111 right here: