# Historical population surface: validation

Generated by `export_surface.py` from the same loader the percentile
table uses, so these totals describe the exact points behind the
published numbers.

## National totals against the published census

Coverage differs by year, so England and Wales and Scotland are shown
separately rather than as a Great Britain figure that would hide which
nations a year contains. A year covering one nation is compared only
against that nation.

| year | coverage | points | built | published | error | E&W built | E&W error | Scotland built | Scotland error |
|---|---|---:|---:|---:|---:|---:|---:|---:|---:|
| 1801 | GB | 290,358 | 10,439,480 | 10,500,956 | -0.6% | 8,833,615 | -0.7% | 1,605,865 | -0.2% |
| 1811 | GB | 295,023 | 11,919,933 | 11,970,120 | -0.4% | 10,123,404 | -0.4% | 1,796,529 | -0.5% |
| 1821 | GB | 296,912 | 14,004,669 | 14,091,757 | -0.6% | 11,916,040 | -0.7% | 2,088,629 | -0.1% |
| 1831 | GB | 300,245 | 16,229,360 | 16,261,183 | -0.2% | 13,868,003 | -0.2% | 2,361,357 | -0.1% |
| 1841 | GB | 375,979 | 18,522,947 | 18,534,332 | -0.1% | 15,904,724 | -0.1% | 2,618,223 | -0.1% |
| 1851 | GB | 222,691 | 20,820,852 | 20,816,351 | +0.0% | 17,936,417 | +0.0% | 2,884,435 | -0.1% |
| 1861 | GB | 222,988 | 23,129,635 | 23,128,518 | +0.0% | 20,071,335 | +0.0% | 3,058,300 | -0.1% |
| 1871 | GB | 223,049 | 26,073,232 | 26,072,284 | +0.0% | 22,716,934 | +0.0% | 3,356,298 | -0.1% |
| 1881 | GB | 223,266 | 29,688,428 | 29,710,012 | -0.1% | 25,956,246 | -0.1% | 3,732,183 | -0.1% |
| 1891 | GB | 223,164 | 33,043,960 | 33,028,172 | +0.0% | 29,021,374 | +0.1% | 4,022,586 | -0.1% |
| 1901 | GB | 222,739 | 36,971,775 | 36,999,946 | -0.1% | 32,530,702 | +0.0% | 4,441,074 | -0.7% |
| 1911 | GB | 220,718 | 40,615,968 | 40,831,396 | -0.5% | 35,883,939 | -0.5% | 4,732,029 | -0.6% |
| 1921 | GB | 586,533 | 42,738,885 | 42,769,196 | -0.1% | 37,860,056 | -0.1% | 4,878,829 | -0.1% |
| 1931 | GB | 585,406 | 44,838,631 | 44,795,357 | +0.1% | 39,999,100 | +0.1% | 4,839,531 | -0.1% |
| 1951 | GB | 271,380 | 48,896,416 | 48,854,303 | +0.1% | 43,803,867 | +0.1% | 5,092,549 | -0.1% |
| 1961 | GB | 271,378 | 51,342,692 | 51,283,892 | +0.1% | 46,166,986 | +0.1% | 5,175,706 | -0.1% |
| 1971 | GB | 229,755 | 53,949,482 | 53,978,540 | -0.1% | 48,723,984 | -0.1% | 5,225,498 | -0.1% |
| 1981 | GB | 232,273 | 53,552,940 | 53,556,911 | -0.0% | 48,521,377 | -0.0% | 5,031,563 | -0.1% |
| 1991 | GB | 302,955 | 54,657,745 | 54,888,840 | -0.4% | 49,663,657 | -0.5% | 4,994,088 | -0.1% |
| 2001 | GB | 305,563 | 57,103,927 | 57,103,927 | +0.0% | 52,046,327 | +0.0% | 5,057,600 | -0.1% |
| 2011 | GB | 318,489 | 61,371,315 | 61,371,315 | +0.0% | 56,080,489 | +0.0% | 5,290,826 | -0.1% |
| 2021 | GB | 332,079 | 65,036,876 | 65,034,142 | +0.0% | 59,602,309 | +0.0% | 5,434,567 | -0.0% |

## Placement error, measured

Three separate measurements, none of them assumed.

**Modelled years, 1851 to 1961.** Error profile measured against 1971
enumeration district ground truth and scaled by each year's
population-weighted unit width. The relationship is linear at about
0.075 mile of error per mile of unit width, across an eightfold range.

**The 1801 to 1841 tier** is placed differently, 71% of its
population by parish name against GB1900 (inside the registration
district the 1852 index gives the name, where it gives one) and the
rest over county label density, so the width model does not
describe it. Two measurements instead. The county-density fallback:
1901 England and Wales, whose positions are sound, re-placed with the
same share by county density.

| percentile | p50 | p75 | p90 | p95 | p99 |
|---|---:|---:|---:|---:|---:|
| miles | 0.09 | 0.17 | 0.24 | 0.30 | 0.40 |

And the name placement itself. The 1852 Index to the Census of 1851
says which registration district every parish lay in. Since
2026-09-13 the placement uses it: a name the index knows is placed
only on a label inside that district (`district` in the
transcription's `match` column), or, where no label inside carries
the name, spread over the district's labels (`district-density`).
Before that constraint, measured against the index, the county-wide
search put 11 to 88% of each mode's population outside the right
district. The constraint means the index can no longer measure the
rows it constrains, and the names it does not know (about 45% of
the named population) still get the county-wide search. So the
name-placement term is estimated (`measure_name_placement.py`,
`comparative1831/name_placement.json`): the constrained rows are
re-placed county-wide, the share that lands outside its district
and the shift in the Great Britain percentiles are measured, and
that shift is scaled from the constrained population to the
unconstrained one. It assumes the unknown names misplace at the
same rate as the known ones; they are the garbled and the odd
ones, so the true rate is if anything higher. The residual of the
constrained rows (a point on the wrong village inside the right
district is not seen) is reported beside it.

| year | named parishes the index knows | county-wide re-placement outside its district | shift p50 / p90 / p95 (mi) | unconstrained share of E&W | term p50 / p90 / p95 (mi) | residual outside after the constraint |
|---|---:|---:|---:|---:|---:|---:|
| 1831 | 6,509 of 11,783 | 530 of 6,066 (9.9% of population) | -0.00 / +0.00 / -0.07 | 32.0% | 0.00 / 0.00 / 0.06 | 30 (0.12% of E&W) |
| 1841 | 6,187 of 11,356 | 550 of 5,748 (8.7% of population) | +0.03 / +0.00 / +0.04 | 32.6% | 0.03 / 0.00 / 0.03 | 35 (0.17% of E&W) |

The district-density rows (a parish the index places but no label
names, spread over its district's labels) carry a spread error of
their own, measured on the constrained rows by moving each to the
mean of its district's labels and scaling the shift to the
district-density share:

| year | share of E&W on district density | displacement p50 / p90 (mi) | term p50 / p90 / p95 (mi) |
|---|---:|---:|---:|
| 1831 | 5.2% | 3.3 / 7.1 | 0.04 / 0.01 / 0.15 |
| 1841 | 5.7% | 3.3 / 7.4 | 0.04 / 0.03 / 0.02 |

Both terms are carried in the percentile table for 1801 to 1841
(1801 to 1821 share 1831's placement); against 1831's p50 of 49
miles the fallback term is under half a percent.

**Excluding Scotland**, measured at 1901 and 1911, the only modelled
years holding both nations: p50 moves 0.01 mile, p90 0.07, p99 1.1 to
1.5. Medians are therefore comparable across the whole run and the
tails of the England-and-Wales-only years are understated, because
Scotland is where the far-from-a-station population lives.

**England and Wales 1851 to 1901** place each registration district's
people by registration sub-district (`build_rsd_points.py`). The
sub-districts have counts and printed acreage but no geometry, so
each is seeded at the 1841 parish point or GB1900 label inside its
district that carries its name, claims the district's 1841 parishes
nearest to it up to its printed acreage, and has its count spread
over the settlement cells of that territory. Measured with the 1841
parish points re-spread both ways against the 1851 stations, the
p90 placement error falls from 0.48 to 0.31 mile and the p50 from
0.17 to 0.11; the finest partition this spreading could have, every
parish over its own cells, sits at 0.14. The unit-width term in the
percentile table follows the tier's own cell widths.

## Known gaps

- **1841 England and Wales is read from its own volume** (2026-09-11).
  The Enumeration Abstract's page images were read cell by cell, and a
  row stands only when its printed males and females sum to its total:
  17,191 parish, township and hamlet rows pass, 54 of 55 counties are
  anchored to the volume's own summary of England and Wales, and the
  national total lands within 0.1% of the census. About a fifth of the
  rows, garbled names and London parishes without a map label, sit on
  county label density as in 1831. Until this build the year carried
  1831's shape scaled to the 1841 total, at a measured p50 cost of
  0.45 mile; its band is now the early tier's. Scotland 1841 is its
  own data.
- **England and Wales 1921 and 1931 sit on civil parishes inside rural
  districts** read from Table 3 of the census County Reports on
  histpop.org (BUILD_EW_1921_1931.md). A row counts only where
  persons equals males plus females, a district only where its rows
  sum to the printed total (or one row is fixed by subtraction from
  it, or the sum is within 1% of the district tier's estimate): 356
  of 589 rural districts in 1921 and 484 of 602 in 1931, 9% and 14% of
  the population, the rest of it in boroughs and urban districts or
  in the districts that did not prove, which keep district rows. On
  the replaced districts, 1841 parish points re-spread both ways
  against the year's stations put the district method 0.32 mile
  above truth at p90 and the parish method 0.08, against a floor of
  0.08; the placement band for both years follows the tier's own
  cell widths. 1951 and 1961 stay on districts: no open digitisation
  of their County Reports exists.
- **Scotland 1921 to 1971 now rests on NRS Table 8** (1971 County
  Report, population by civil parish 1801-1971, supplied by NRS on
  request, publishable with credit), extracted from image-only scans.
  The backbone is each year's Scottish census total apportioned by
  parish shares interpolated log-linearly between the exact 1911
  figures and the 1981 surface; OCR'd parish cells override the
  backbone only where the row read its seven already-known columns
  correctly and the value is plausible. Every year is now GB.
- **1971 Scotland is placed on the 1981 surface, not its own.** Its
  ED codes carry the 37 PRE-1975 counties while 1981 uses the
  post-1975 regions and districts, sharing no prefix, so the 1973
  reorganisation makes individual 1971 EDs unplaceable. Its census
  total is spread over the 1981 Scottish surface instead, meaning
  Scotland's internal distribution in 1971 is 1981's, scaled. Both
  worries were priced where truth is known: placing Scotland as one
  unit rather than at ED level costs +0.07 mile at p90, and using a
  twenty-year-stale density basis costs -0.01, against -0.22 at p90
  and -2.10 at p99 for leaving Scotland out. Coarse and stale both
  beat absent by a wide margin.
- **1981 was the worst year in the table until 2026-09-11**: the
  count file's own England and Wales total was 47.9M against a
  published 49.2M because the 1981 table splits its population base
  (present residents in one column, absent residents in another) and
  only the first was read; both are now summed, and Scotland 1981 is
  calibrated to the 1991 Monitor (`calibrate_scot_1981.py`).
- **Scotland 1981 is now placed** (`build_scot_1981.py`), though no
  Scottish 1981 boundary or centroid product exists anywhere. A
  Scottish 1981 ED code is region, district, ward, ED, and Scotland's
  1991 output area codes reuse it with a split suffix, so 88.7% of
  1981 EDs have a 1991 output area sharing their whole eight-character
  code and can be placed on their own successor. Verified rather than
  assumed: across 15,747 matched pairs the median 1991-to-1981
  population ratio is 0.988, and Scotland fell about 1% that decade.
  98.0% of the Scottish 1981 population places at ED level, 2.0% at
  ward, 0.1% at district, and 1.1% not at all.
- **1991's p90 sits above both 1981 and 2001 despite more stations
  than 1981, and this is NOT a placement artefact.** It was attributed
  to plain centroids; it is not. With E&W spread over 2001 centroids
  inside each 1991 polygon p90 reads 4.56, over 1981 centroids 4.55,
  on plain centroids 4.57. It lives in the counts across districts,
  the file total reconciles to the census within 0.1%, and with
  today's population held fixed the station workstream's p90 falls
  monotonically through 1991. What remains is a real 1981-to-1991
  population shift, consistent with that decade's counter-
  urbanisation; no evidence of a data defect was found.
- **Years 0.4% or more from the census**, by nation, from the table above:
- **1801 reads 0.6% low** (England and Wales -0.7%, Scotland -0.2%). Candidate cause: the Comparative Account's parish rows do not reach every person the census summary counted (rows the reader could not prove are dropped, and the volume's own county totals are the anchor).
- **1811 reads 0.4% low** (England and Wales -0.4%, Scotland -0.5%). Candidate cause: as 1801.
- **1821 reads 0.6% low** (England and Wales -0.7%, Scotland -0.1%). Candidate cause: as 1801.
- **1911 reads 0.5% low** (England and Wales -0.5%, Scotland -0.6%). Candidate cause: both nations' 1911 files sum under the census (England and Wales on the SN 9035 local government district file, Scotland on the SN 9180 parish series); not attributed further.
- **1991 reads 0.4% low** (England and Wales -0.5%, Scotland -0.1%). Candidate cause: the 1991 count file is the enumerated population by enumeration district, before the census's own imputation for under-enumeration.

## Files

In the published folder beside this file:

- `population_surface_early.csv.gz`, `population_surface_victorian.csv.gz`,
  `population_surface_interwar.csv.gz`, `population_surface_modern.csv.gz`:
  every year's points, split by era.
- `distance_percentiles.csv`: the distance-to-station series by
  census year with the modelled band per percentile, the direct 1841
  check where a tier has one, and the missing-station sensitivity
  (series.py; the same numbers as distance_percentiles.md in the
  pipeline).
- `transcription_ew_parish_1801_1831.csv`, `transcription_ew_parish_1841.csv`,
  `transcription_ew_parish_1921.csv`, `transcription_ew_parish_1931.csv`,
  `transcription_scot_parish_1921_1971.csv`: the rows read from the
  printed volumes, with the proof each row passed (README).
- `SHA256SUMS`: checksums of these files and of the four era
  GeoPackages (`population_surface_early.gpkg` and so on, one layer
  per census year), which are kept in the project archive.

In the pipeline: the unit polygons with counts for the three modelled
tiers (registration districts 1851-1911, local government districts
1911-1971, Scottish parishes 1801-1891) and the station table's pin
(hash and row count of the table these numbers were computed against).

## How each year is built

- **early**: parish, 1831 Comparative Account OCR, placed by GB1900 name with a county-density fallback, anchored to printed county totals
- **rsd**: registration district, distributed over GB1900 label density
- **lgd**: local government district, distributed over GB1900 label density
- **scot_parish**: Scottish civil parish, distributed over GB1900 label density
- **official**: official population-weighted centroids, no modelling