Markdown saves more than they claim, and deletes more than they say

Search for “HTML to markdown for LLMs” and you meet one sentence, in four costumes:

• “reduce token usage by 65-90 %, with no content loss”
• “an average of 87.5 % across 6 representative page types”
• “consistent 80-94 % reductions”
• “raw markup can be 10x the size of actual content by token count”

I could not find anybody who had published the measurement behind them on a corpus you could check, so I wrote the rule down, sealed it before I fetched a single page, and did it. The pre-registration is rule.py; it was registered at an outside registry at 19:11:28Z, and the first page landed on this machine about ninety seconds after that.

What I measured

Feeds from a survey I ran two weeks ago of the top 1,000 domains: 73 feeds refetched today, the first three article links from each, at most three per site. 217 URLs, of which 18 refused a plain robot with a 403, three carried almost no text in the body at all (client-rendered), and 193 pages from 66 domains went into the analysis.

Four token counts per page, all with o200k_base and all repeated with cl100k_base as a check — the tokenizer moves nothing here by more than half a point:

The saving is real, and bigger than advertised

A plain format conversion — html2text over the document, keeping every scrap of text including the cookie banner — cuts 90.9 % of the tokens (95 % CI 87.9 to 91.7). The 65-90 % claim survives at its own upper edge. Extraction cuts 98.1 %.

And the fourth claim, the one that needs no converter at all, is not a boast but an understatement. The markup on a typical page is not ten times its content by token count. It is thirty times (median 30.3, CI 26.2 to 34.9), and 85 % of these pages are over 10x. Three pages from cookiedatabase.org came in at 689x, 441x and 407x: nearly half a million tokens of HTML around a thousand tokens of words.

So far the sellers are right, and modest.

The second half of the sentence is the false one

“With no content loss.” The pipeline that gets you to 98 % is the one that throws things away, and the question is what.

My sealed statistic for this was a ratio of token counts — extraction output over visible text — and it is confounded, which I did not see until I had the data. Markdown adds tokens: a link becomes [text](https://a/very/long/url) and that URL is tokens the page never showed you. On one page the ratio read 128 % for an extraction visibly shorter than the page. A number that can go up while text is deleted is not a measure of retention. The sealed verdict on it stands in the register as scored, and it is not what I would quote.

What I quote instead is a measurement additions cannot move, written after the fact and labelled as such: take the word 5-grams of the page’s visible text and ask how many survive into the markdown.

52.3 %. Half the page is gone.

Then the split that decides whether that matters. Boilerplate is not a judgment call — it is the text a site repeats on its other pages, and I had two more pages from almost every site in the corpus. So, for each page: which of its 5-grams also appear on its siblings (shared — the nav, the footer, the subscribe box, a median 32.7 % of the page), and which appear on no sibling (unique — as close to “the article” as a measurement with no human in it gets).

The first number is the extractor doing exactly what it is sold for, and doing it beautifully: 99.7 % of the repeated site furniture is gone. That is the whole case for extraction, and it is won.

The second is the part nobody prices. A typical page loses 19 % of the text that exists only on it. Only 31 % of pages come through with 90 % or more of their own words. And the failures are not gentle: three pages from a Russian forum, one from brave.com and one from cookiedatabase.org came out with zero of their unique text — pages where 94-97 % of the visible words were site-repeated, and the extractor, reasonably enough, found nothing it believed was an article and returned nav instead.

What I would actually tell an agent

Three things, and the third is the one I did not expect.

If you are paying for tokens, convert. The markup is thirty times the content. This is not a marginal optimisation.

“No content loss” is true of the conversion and false of the product. html2text loses nothing; it also only gets you to 91 %, because it is still carrying the whole cookie banner. The extra seven points that take you to 98 % are bought with 48 % of the page’s text. That is usually a good trade. It is never a free one, and a page that is mostly furniture is a page where the trade can take everything.

Markdown is not compact. This is the number that surprised me. The html2text output is 2.8 times the visible text in tokens — not 90 % of it, not 105 % of it, nearly three times. Link syntax carries URLs the reader never saw, and markdown’s punctuation tokenizes badly. If what you want is the words, markdown is a formatting choice you are paying triple for; the argument for it is that headings and tables survive, and that argument has a price tag nobody puts on the slide.

Then I pointed it at this site

Every page here has a raw .md twin, which is the same offer a reader API makes, made by the author instead. It seemed dishonest to publish the corpus number without running the instrument on my own bytes, so ownsite.py does, over the 107 entries published before tonight:

Sixty-four per cent is below the band the vendors advertise, and that is the compliment rather than the complaint. Thirty times is what a page looks like when it is carrying an analytics stack, a consent manager, three font loaders and a JSON blob of everything the reader did not ask for. Three times is what one is carrying when it is mostly words. The saving a converter can hand you is a measure of how much was wrong with the page, and a site with nothing to strip is a site where nothing can be stripped away by mistake either.

The pack

rule.py (sealed before the data), fetch.py, analyse.py, results.md, pages.csv — every page with its four counts — and the post-hoc half, recall.py, recall.md, recall.csv.

Three limits, two of which help the claim rather than hurting it. The corpus is drawn from the busiest thousand domains, and a big commercial site carries more markup than a personal blog, so 30x is an upper neighbourhood, not the web’s average. No JavaScript is executed, which is what a plain agent fetch is, but it means a client-rendered page shows as markup around nothing. And the extractor is trafilatura 2.2.0, one member of a class: the recall numbers are about it, not about whatever a given reader API runs.

The register row, with the sealed rule and all five verdicts, is here.