"""The verdicts for "Wikipedia articles lead to Philosophy", two claims at once.
Committed 2026-09-15 (session 62) before any chain was followed.
The page Wikipedia:Getting_to_Philosophy, read that evening, says two things
that cannot both be true. Its short description: "Wikipedia phenomenon where
~97% of all articles link to Philosophy" (the figure is Lamprecht et al. 2016).
Its body, since a July 2026 edit and without a source: "some first links were
changed or removed in July 2026. As a result, almost no articles continue to
lead to 'Philosophy'."
The sample. 500 articles from the English Wikipedia's own random generator
(`list=random`, namespace 0, redirects excluded), drawn in batches at run time.
Disambiguation pages count as articles, as they do for the generator.
One step, `first_link(html)`. The page is the parser output of the article's
current revision (`action=parse`, `prop=text`, `redirects=1`). The link taken is
the first one, in document order, that satisfies every line below:
* it sits inside a ,
or that is a direct child of
div.mw-parser-output (so infoboxes, tables, hatnotes, figures, navboxes,
sidebars and blocks inside other divs are never read);
* it is not inside parentheses — a depth counter over the block's text,
starting at 0 for every block, links' own text included;
* it is not inside , , , or (bold is allowed),
nor inside any element whose class names IPA, reference, noprint or
mw-ref, or whose id is coordinates;
* its href starts with /wiki/, its class is not `new` (a red link), and the
title is not in another namespace (a prefix before ':' that is one of
NAMESPACES below);
* the title, fragment removed and underscores read as spaces, is not the
page's own title.
The title is then fetched with redirects=1, and the resolved title is the next
step.
A chain from a start article ends at the first of:
philosophy the resolved title is "Philosophy" (the start itself included);
loop the resolved title was already seen in this chain;
dead_end no link satisfies the rule, or the page does not exist;
cap 200 steps without any of the above.
Every title is fetched once per run and its first link cached, so all chains
in a run see the same encyclopedia.
The share. p = philosophy / 500, with a Wilson 95 % interval (lo, hi).
Verdict on claim A, "about 97 % of articles lead to Philosophy":
survived lo >= 0.90 (nearly all, as the claim means it);
killed hi < 0.90;
inconclusive otherwise.
Verdict on claim B, "almost no articles continue to lead to Philosophy":
survived hi <= 0.10;
killed lo > 0.10;
inconclusive otherwise.
Reported with the verdicts, and not scored: the outcome counts, the median
chain length of the philosophy chains, and the titles that the loops end in.
Anything beyond that is labelled after the data.
"""
import math
import re
from lxml import html as lxml_html
N = 500
CAP = 200
A_BAR = 0.90
B_BAR = 0.10
NAMESPACES = {
"talk", "user", "user talk", "wikipedia", "wikipedia talk", "wp", "file",
"file talk", "image", "mediawiki", "template", "template talk", "help",
"help talk", "category", "category talk", "portal", "portal talk", "draft",
"draft talk", "timedtext", "module", "module talk", "special", "media",
"wiktionary", "wikt", "wikisource", "s", "wikiquote", "q", "wikibooks", "b",
"wikinews", "n", "wikiversity", "v", "wikivoyage", "voy", "commons", "c",
"species", "meta", "m", "mw", "d", "wikidata", "foundation", "wmf",
}
SKIP_TAGS = {"i", "em", "sup", "sub", "small"}
SKIP_CLASS = re.compile(r"\b(IPA|reference|noprint|mw-ref)\b")
def _norm(title):
title = title.split("#", 1)[0].replace("_", " ").strip()
return title[:1].upper() + title[1:] if title else title
def _link_title(href):
if not href or not href.startswith("/wiki/"):
return None
from urllib.parse import unquote
title = _norm(unquote(href[len("/wiki/"):]))
if not title:
return None
if ":" in title and title.split(":", 1)[0].strip().lower() in NAMESPACES:
return None
return title
def first_link(page_html, own_title):
"""The next title under the rule, or None."""
root = lxml_html.fromstring(page_html)
outs = root.xpath('//div[contains(concat(" ", normalize-space(@class), " "), " mw-parser-output ")]')
if not outs:
return None
own = _norm(own_title)
for block in outs[0]:
if not isinstance(block.tag, str) or block.tag not in ("p", "ul", "ol"):
continue
found = _scan(block, own)
if found:
return found
return None
def _scan(block, own):
"""First qualifying link in one block; parentheses counted over all text."""
depth = 0
def walk(el, skipped):
nonlocal depth
if not isinstance(el.tag, str): # comments, processing instructions
return None
cls = el.get("class") or ""
skip = (skipped or el.tag in SKIP_TAGS or bool(SKIP_CLASS.search(cls))
or el.get("id") == "coordinates")
if el.tag == "a" and not skip and depth == 0 and "new" not in cls.split():
t = _link_title(el.get("href"))
if t and t != own:
return t
depth = _count(el.text, depth)
for child in el:
r = walk(child, skip)
if r:
return r
depth = _count(child.tail, depth)
return None
return walk(block, False)
def _count(text, depth):
if text:
for ch in text:
if ch == "(":
depth += 1
elif ch == ")" and depth > 0:
depth -= 1
return depth
def wilson(k, n, z=1.96):
p = k / n
d = 1 + z * z / n
c = (p + z * z / (2 * n)) / d
h = z * math.sqrt(p * (1 - p) / n + z * z / (4 * n * n)) / d
return p, c - h, c + h
def verdicts(k, n=N):
p, lo, hi = wilson(k, n)
a = "survived" if lo >= A_BAR else "killed" if hi < A_BAR else "inconclusive"
b = "survived" if hi <= B_BAR else "killed" if lo > B_BAR else "inconclusive"
return {"p": p, "lo": lo, "hi": hi, "A": a, "B": b}