Wikipedia link graph · exact single-source shortest paths
Graph Surprisal
Two articles can mean nearly the same thing and still sit far apart in Wikipedia's link graph. This page lists those pairs and calls them Surprisal Connections. Distance is measured in bits — edge weight is log2(out_degree(u)), so the length of the shortest path is its search information: the negative log probability that a reader clicking uniformly at random follows it. A long distance means the encyclopedia makes the connection hard to find. Each Surprisal Connection is a candidate, not a verdict.
Every number on this page is invented sample data, written by hand so the interface could be built and checked before the real index existed. The titles are real Wikipedia articles; the bits, hops, cosines and page ids are not. Do not cite anything here.
This is a shakedown set of ~3,100 source articles that were chosen as benchmark endpoints, not as a computer-science corpus. Expect the results to look odd — that is the expected outcome of running the app against whatever had already been swept, and it is the point of doing it before spending compute. The ~10,000-article computer-science slice is still being computed; it is a multi-day CPU job.
The app can only answer for articles it has swept. Every swept source is listed below. If an article is not in that list, this page has nothing to say about it — that is an absence of data, never evidence that no Surprisal Connection exists.
Loading index…
Swept sources
Plain case-insensitive substring match over swept titles. No semantic search yet — a word that does not appear in a title will not match it. The number beside each title is that source's Surprisal Connection count.
Arrow keys move, Enter opens. Columns: Surprisal Connections · out-degree.
Surprisal Connections
Pick a source article from the list, or search for one. You will get its Surprisal Connections — the articles closest to it in meaning that are farthest from it in the link graph — each with the distance in bits, the hop count on that path, the cosine similarity, and whether a direct link exists.
How to read the numbers
- bits — search information
- The exact shortest-path distance from the source to the partner, under edge weight w(u→v) = log2(out_degree(u)). That quantity is search information — the literature's name for it (Rosvall 2005, Sneppen 2005), and the name the project stores it under (search_information_log2_degree). It is the negative log probability that a reader clicking uniformly at random among a page's links follows the best path. "Surprisal" is this page's name for the same number, not a second quantity. Bigger is more surprising. For scale: a random article sits about 32 bits from a typical source (the median source's median), and 46 bits is that source's own 99th percentile. The sliders open at the cut the index was built with.
- probable noise, dropped
- Some pairs clear both cuts only because their titles look alike, so the build drops them by title rule and counts what it dropped: a title of four characters or fewer (acronyms); both titles carrying a year (1927 in Wales → 1880s in Wales); both titles starting with a number (military units, chemical names); and a partner whose title is a whole run of the source's (Albert III of Saxony → Albert III, usually a disambiguation page); and both titles comma-qualified (Abbot, Maine → Meadow, Texas — settlement stubs written from one template). These are heuristics on titles alone, so they also drop some real pairs; each source shows how many it lost.
- hops
- How many links lie on that minimum-weight path. This is not the minimum click count: a shorter click path may exist through higher-degree pages and cost more bits. Ties are broken lexicographically on (distance, hops). A blank hop count means the stored sweep recorded no path — the partner was not reached.
- cos
- Cosine similarity between the two articles' bge-m3 embeddings. High cosine can also mean the two articles are the same kind of article rather than about the same thing — a known false-positive mode, and a reason to read the pair rather than trust the number.
- direct link
- Whether the source links straight to the partner in this dump. A direct link makes the distance small by construction, so at a high bits threshold this is almost always blank.
- direction
- Every distance here is d(source → partner). The reverse direction is a different number and needs the partner to have been swept too, which it usually has not been. This page claims nothing about mutual reachability.