Posts for: #Content-Negotiation

HTTP Already Knows How to Serve AI Agents. We Just Never Turned It On.

HTTP Already Knows How to Serve AI Agents. We Just Never Turned It On.
Update — less than an hour after publishing this article, my Hermes Agent Jasper and I fixed it. This site now serves both /llms.txt and per-page markdown via Accept: text/markdown. See Well, That Escalated Quickly: Making Your Hugo Site LLM-Friendly in Under an Hour for the how-to.

Every time an AI agent reads a page on your website, it burns four out of every five tokens on scaffolding it cannot use. The <div> wrappers, the navigation menus, the cookie consent banners, the tracking scripts, the CSS class names, all of it takes up space in a context window that could have held your actual content. And context windows are finite. Models that claim 200,000-token capacity become unreliable around 130,000 per AIMultiple’s January 2026 analysis. When you send HTML, you are burning half your usable context on markup the agent cannot even use.

[Read more]