The Web Is Growing A Second Layer – Almost A Third Head

The previous couple of weeks have been noisy. Google shipped one thing known as the Open Knowledge Format. Then Google Builders introduced the Agentic Resource Discovery (ARD) specification.

In the meantime, each search engine optimisation LinkedIn feed is lit up with somebody both declaring markdown the way forward for the online or explaining why you need to ignore all of it.

The reality, as per standard, sits someplace extra attention-grabbing than both camp.

The online is creating a parallel machine-readable infrastructure (MCP/WebMCP, OKF, ARD, LLMs.txt…) and SEOs who perceive what every layer really does, relatively than treating all of it as “AI search engine optimisation” or a silver bullet, will make higher choices about the place to spend their time.

First: The Layer Cake

There are at the least six distinct issues being mentioned underneath the umbrella of “making your website AI-ready.” They sit at totally different layers and serve totally different functions:

• Crawlable HTML Pages: Nonetheless the inspiration. Nothing has modified right here. Every part else sits on prime.
• Schema.org/Structured Information: Semantic hints baked into HTML that inform machines explicitly what a web page is about. It’s, in essence, a vocabulary.
• LLMs.txt: Primarily a navigation file. Its goal is to basically inform an AI agent that’s already in your website which pages matter. However as John Mueller places it on the Search Off the Record podcast:

“If somebody is already in your web site, perhaps some sort of automated system is useful. The place if it goes, I need to go to Martin’s Splitt and purchase {a photograph}, then the LLM system can go to your web site and might go searching, like, how do you purchase {a photograph}? Perhaps he has some tips for me as an agent for getting pictures. That sort of is smart.”

• MCP/WebMCP: Earlier than ARD got here into play, we had been introduced with one other answer for the problem of interoperability. An MCP, in its easiest rationalization, is a standardized method for an AI to connect with your providers to extract information or take motion. WebMCP, because the title itself suggests, offers web sites a strategy to have interaction with brokers immediately. WebMCP is for dwell browser interactions on a webpage; MCP is for instruments and providers past the web page.
• Open Information Format (OKF): A bundle of markdown recordsdata with YAML frontmatter.
• Agentic Useful resource Discovery (ARD): A brand new open spec for the way brokers discover and confirm instruments, expertise, and different brokers throughout the online. Right here, the main target isn’t your content material; it’s your capabilities.

For ecommerce, there’s one other layer value naming individually – the product feed – fairly probably the future of retail discovery.

Every layer does one thing totally different.

I may hold including to this record; there’s a brand new layer popping up each 5 minutes. I’m stopping right here. It’s ballooning.

What OKF Really Is (And Isn’t)

Google published the OKF spec quietly, bolted to a rebrand of Dataplex into Information Catalog.

The format itself is nearly disarmingly easy: a listing of markdown recordsdata, every with a small YAML header declaring a sort, title, description, useful resource, and a few tags. The recordsdata hyperlink to one another like all markdown doc would. That’s it.

As Google’s personal weblog places it, OKF is “simply markdown, simply recordsdata, simply YAML frontmatter.”

search engine optimisation Suganthan Mohanadasan has a clear breakdown of this. He describes OKF as one ground in a stack that now contains sitemap.xml (which URLs exist), LLMs.txt (which pages you most need learn), and OKF (the library itself). They stack relatively than compete.

The confusion units in not once you take a look at what OKF is, however what it does and by which layer of the agentic and search mayhem it sits.

In my thoughts, OKF isn’t a retrieval system. It doesn’t substitute crawling. And, personally, I don’t see a future the place AI methods not ingest huge quantities of HTML or the place search and RAG aren’t a multistep advanced pipeline that consists of self-reported and “unbased” alerts.

Any self-reported system can and will likely be gamed. So pondering you may simply slam a bunch of markdown recordsdata in your website and be THE most well-liked selection in retrieval and discovery is far-fetched.

OKF is a higher-signal supply amongst many. It could cut back parsing value and enhance sign high quality, but it surely doesn’t substitute current pipelines.

It’s additionally value being sincere right here: OKF was constructed for knowledge groups, not advertising and marketing websites.

It arrived as a strategy to share inner information, i.e., desk schemas, runbooks, metric definitions, between AI brokers inside organizations. Pointing it at a public web site to me appears a bit like we’re but once more repurposing.

Francois Vanderseypen makes essentially the most exact level about what OKF actually is and isn’t: a directed graph of markdown recordsdata is an online of paperwork, not a information graph (at the least not in its purest sense). An actual KG has specific, queryable, typed relations.  OKF leaves what a hyperlink implies solely as much as the producer, and an LLM nonetheless has to deduce the semantics each single time it reads it.

For me, this factors to the crux of how I perceive the online and what we do as SEOs. OKF doesn’t change the stack. It provides yet another enter into it.

It’s not a shortcut. There aren’t any shortcuts.

The Schema.org Parallel, And Why It Issues

One of many patterns to know right here is the one Schema.org already went by means of.

Structured knowledge adopted a predictable arc:

Adoption – rating enhance – widespread use (and gaming) – platform studying – lowered dependency as a rating sign.

FAQ schema had a second in SERPs, then Google discontinued the FAQ rich result. The platforms be taught from the alerts, fold the teachings into the algorithm, and the express markup turns into much less obligatory.

OKF and LLMs.txt might comply with the identical path. They’re most useful early, as clear alerts in a world the place AI methods are nonetheless studying to parse the online.

Over time, if the codecs work, the methods be taught. Specific markup turns into redundant or stays a verification layer. For instance, in ecommerce, specifically, schema and feed alignment has turn into increasingly more vital. One other notch within the name for co-ownership of the product feed between SEO and paid teams!

There’s additionally a subtler level value making right here in regards to the relationship between schema.org and discovery. Jarno van Driel’s deep dive on product variants in Search Engine Journal illustrates this effectively: For years, Google Search and Google Service provider Heart had conflicting structured knowledge necessities, forcing publishers to duplicate markup. Schema.org evolves to shut gaps, but it surely’s sluggish, it’s advanced, and implementation remains to be usually a large number.

Structured knowledge has by no means been a plug-and-play rating lever. OKF gained’t be both.

Ought to You Convert Your Web site To Markdown?

It’s a giant fats no from me. That doesn’t imply I gained’t check it and apply fastidiously!

And John Mueller stated it on the Search Off the Record podcast:

“Relating to issues like a search engine or most likely additionally in generic LLM system, having a web site that makes use of regular HTML for the pages is important. As a result of a search engine or crawler can simply go to that web page. It could actually recognise all the different hyperlinks which might be throughout the web site.”

The structural info in HTML – nav hyperlinks, footers, header hierarchies, inner hyperlinks – is how crawlers perceive your website’s form. Markdown recordsdata strip all of that out. You’d be breaking discovery so as to marginally enhance machine readability of particular person pages.

Just lately, on LinkedIn, I even noticed a piece of research exhibiting how “Your navigation could be consuming your LLM (it’s ChatGPT Deep Analysis in truth) studying funds.” Fascinating findings, however please don’t take away your navigation to “avoid wasting tokens”!

Jono Alderson makes this point brilliantly: “A web page is not only a container for phrases. It’s an editorial artifact.” Hierarchy, emphasis, placement, what comes first, what’s outstanding, what’s tucked in a footnote … these aren’t fairly decorations for people. “They’re alerts about which means.”

“While you flatten a web page into markdown, you don’t simply take away muddle. You take away judgment, and also you take away context.” And the second you publish a machine-only illustration, you’ve created a second candidate model of actuality.

The boring repair nonetheless works: Semantic HTML, clear construction, wise hierarchy, content material that exists when the web page hundreds.

John Mueller covers the markdown debate extensively within the podcast: The parallel variations downside, the dynamic rendering classes we already discovered the exhausting method, and why sustaining a shadow model of your website for AI doubles your upkeep burden and creates a debugging nightmare no one will inform you about.

The one exception Mueller carves out is developer documentation:

“You probably have one thing like developer documentation, the place, once more, if the agent or the LLM system already is aware of about your web site and the consumer says, how do I usethis API? Then for those who give the LLM system a Markdown file, it’s quite a bit simpler for it to know.”

Now, I can positively see an easy use case there.

What ARD Is Really Doing

The Agentic Resource Discovery specification, introduced by Google on June 17, 2026, is a special beast solely. It arrived solely a few days behind OKF, not a coincidence, and is already making large waves.

The issue ARD solves is a coordination one. Proper now, an agent must be wired to every instrument, MCP server, or API it makes use of earlier than it may do something with it.

That works once you’re connecting a handful of identified providers. It stops scaling the second the variety of out there capabilities grows past what any crew can pre-configure by hand.

ARD strikes that discovery out of setup and into runtime. The agent finds what it wants when it wants it, relatively than solely realizing what it was advised about upfront.

It’s constructed on two primitives:

• Catalogs: An ai-catalog.json file hosted in your area, describing your out there capabilities (MCP servers, A2A brokers, OpenAPI instruments). Possession of the area acts because the cryptographic basis for identification and belief.
• Registries: Engines like google for the agentic net. They crawl catalogs, index them, and return matching capabilities with the metadata wanted to confirm the writer earlier than connecting.

If OKF is about packaging information for consumption, ARD is about promoting capabilities for connection.

These are parallel efforts at totally different layers of the rising agentic stack. Each shipped inside inches of one another and now adopted with the velocity of sunshine by some very large gamers within the sport, i.e., Hugging Face and their Uncover Instrument.

It’s probably a extra pragmatic wager than the formal logic layer that got here earlier than it and by no means reached net scale. Time will inform.

A Hole Price Watching

Inside days of each specs transport, a contributor opened companion points on the ARD and OKF repos mentioning one thing fundamental was lacking: There’s no agreed media type for an OKF bundle, so a catalog can record one however can’t really acknowledge it as OKF with out sniffing the contents.

Within the meantime, publishers are already promoting bundles in manufacturing utilizing their very own interim varieties, which, as the problem itself notes, gained’t agree with one another.

On the face of it, this seems like a small ask, only a request for a shared label.

After a little bit of a dive into this specific rabbit gap, it seems that’s fairly regular apply. Ready for full settlement earlier than anybody ships something is strictly how a spec dies in committee, and transport quick and patching as actual adoption surfaces is an age-old technique.

Application/json itself wasn’t formally registered until 2006, roughly 5 years after JSON was already in broad, casual use. No person anxious about that, as a result of the price of the label being unsettled was low: A parser would possibly reject one thing or fall again ungracefully.

However OKF is totally different, as a result of what occurs after the fetch is totally different. The artifact behind the label is a bundle an autonomous agent is supposed to ingest, confirm, and probably act on, inside a discovery system constructed particularly for agent-to-agent and agent-to-tool connection. Get the kind unsuitable right here, or go away an agent to deduce it, and the danger isn’t a parse error; it’s a system appearing on one thing it shouldn’t have trusted, with nobody checking the outcome first.

I’m wondering in regards to the danger concerned in settling this later relatively than sooner on this case. I suppose it is determined by how briskly it will get resolved relative to how briskly adoption runs forward of it.

What This Means If You’re An search engine optimisation

Just a few sincere conclusions and my present pondering:

For many advertising and marketing and content material websites, not a lot has modified. HTML, well-structured for people, remains to be the fitting basis. A contact-us type and a clear website structure will serve you higher than any OKF bundle ever will. Discovery nonetheless is determined by hyperlinks, authority, consumer alerts … and indexing.

LLMs.txt is a signpost, not an search engine optimisation instrument. It’s helpful for serving to an agent navigate inside your website as soon as it’s already there. It very possible doesn’t make a giant distinction in how brokers discover you within the first place. And, most likely by no means will.

MCP/WebMCP. Neither is pressing for many advertising and marketing websites immediately, however for those who’re constructing something with programmatic interfaces or ecommerce flows you need brokers to navigate, that is the course the infrastructure is heading.

OKF makes a whole lot of sense for those who’re sitting on structured inner information, i.e., documentation, API references, product specs … and also you need to make it simpler for brokers to eat. The free OKF generator Suganthan constructed will produce a bundle and provide you with a graph view of your inner hyperlink construction as a facet profit. The structural audit alone appears value it. However I will likely be working it on my web site first, not on my shopper’s web site.

ARD is value watching for those who’re constructing providers with programmatic interfaces. You probably have instruments, brokers, or APIs you need discoverable by different brokers, ARD is the rising commonplace for the way that will get executed. Simply know the identity layer beneath it, what an agent is definitely when it finds your catalog entry, remains to be being settled in actual time, so I’d deal with this as infrastructure to look at carefully relatively than construct important paths on simply but.

The schema adoption cycle would possibly repeat. These codecs are most useful now, as early alerts. Implement them if you are able to do it cheaply. Don’t construct your technique round them holding worth ceaselessly and don’t financial institution on them as a silver bullet.

In the end, concentrate on the shiny issues – if your organization has greater fish to fry, i.e., a horrible web site, a model nobody is aware of or cares for, an viewers you don’t perceive … then cope with this primary earlier than you get caught up in any of those new shiny issues.

The Underlying Shift

What all of this factors to is an online that’s genuinely rising a second layer or a 3rd head, one written for machines alongside the one written for browsers and people.

Sitemap.xml advised crawlers which URLs existed. Robots.txt advised them the place to not go. LLMs.txt, OKF, and ARD are related infrastructure for agentic methods: navigation hints, content material packaging, and functionality discovery.

None of it’s necessary immediately. None of it replaces stable HTML, authoritative content material, wise construction, or the factor that really sits beneath all of it: a model value discovering.

However the SEOs who perceive what every layer really does, relatively than treating it as a single undifferentiated “AI search engine optimisation” class, will make higher bets on the place to spend their time.

My cash is on the second layer, a parallel infrastructure written for machines, not a substitute for what already exists.

The third head situation, the place agentic methods totally diverge from the human net, would require a special set of bets than any of us are at the moment making.

Large due to Jarno van Driel, Jono Alderson, Chris Inexperienced, Suganthan Mohanadasan, Kristine Schachinger, Gianluca Fiorelli, Victor Pan, Renee Bigelow (and anybody else I’ve missed) for some good discussions on this subject over the previous couple of weeks.

Extra Sources:

Featured Picture: Collagery/Shutterstock