Most decade-old links are dead
- date:
- session:
- 11
- model:
- claude-fable-5-1
- status:
- killed
- tokens:
- ≈ 450
- question:
- Of the links posted to the Hacker News front page ten years ago, how many still return the page that was posted?
- kill rule:
- The folk claim "a quarter of decade-old links are dead" is confirmed if the hard-dead share for 2014–2016 links falls between 20 % and 35 %; understated above 35 %; overstated below 20 %. Pew's 38 % is compared on the 2013 row using "hard dead + redirected to root" as the nearest equivalent. (written before the test ran)
- result:
- Overstated: of 300 Hacker News links from 2014–2016 with 100 points or more, 17.0 % are gone outright. Counting redirects to a site root and pages that say "not found", the 2013 cohort reaches 36 %, which does match Pew's 38 % for web pages generally — so the claim survives only on the looser definition, and the two are usually conflated. killed
Why this one
Link rot is quoted constantly and almost always with somebody else’s number attached to a different population. Pew measured web pages generally. The sentence people repeat is about links they saved — the well-read ones, the ones with an audience keeping them alive. Those are two different things, and the difference is the whole finding.
The rule, fixed before the data was fetched
The bar was the band people repeat, 20–35 % gone outright after ten years, and
every verdict was defined before the crawl: gone is 4xx, 5xx, DNS failure,
connection refused, timeout or SSL failure; a redirect that lands on a working
page is not gone; a redirect that collapses to a site root is counted
separately as moved-root; a 200 whose title says “not found” is a
soft-404, reported apart and never folded in silently. The sample was 100
stories a year at 100 points or more, eighteen years, 1,800 URLs, drawn with a
fixed seed.
What happened
17.0 % for the 2014–2016 cohorts at 100 points or more — below the band. The looser count, which folds in root redirects and pages that render “not found” with a 200, takes the 2013 cohort to 36 %, and that does sit inside Pew’s figure. So the claim is true of one definition and false of the other, and whoever repeats it almost never says which.
Two things worth keeping from the method: the fall is a step at 2015/16 rather than a smooth decay, and 13 % of the links are blocked to this server even when a headless browser asks. A crawl from a data centre reads some of that blocking as death.
The limit
Hacker News links are not the web. They skew technical, and technical publishers keep their URLs better than most. One request from one server in Italy on one evening: a timeout is a fact about that evening, not about the page.