Capacity planning for maintainers, extrapolated from historical Queenstown measurements. These are planning scenarios, not measured national throughput, current release statistics or a cloud spending estimate.
Use narrow geometry checks before paying for regional rebuilds (reviewer’s guide). For a rebuild, choose incremental, force or yield-only according to the operator guide. The execution comparison explains why the local, district and worker paths are not interchangeable.
The original Queenstown district set was 15,353 buildings (v64 serves
32,838 across Queenstown Lakes). New Zealand has 3,413,097 in the LINZ outlines,
counted cell by cell by tools/plan_national_regions.py: about 222 times
that set. Scaled figures below use that ratio.
| thing | Queenstown | New Zealand (x222) | status |
|---|---|---|---|
| buildings download per street view | 238 kB | same | done: buildings are vector tiles |
| heat map | tiles per region | same per view | done |
data/ on disk with inputs kept |
139 GB | ~31 TB | inputs are deleted after publish, packs kept |
| district build | 6.1 h | ~1,347 VM-hours | regions run in parallel |
| face precompute | 6.6 h (16 cores) | ~1,461 VM-hours | sharded (--shard i/n) |
| region definitions | hand-written | 1,384, derived | done |
Regions are derived. tools/plan_national_regions.py grids the country,
asks LINZ how many buildings each cell holds, and splits any over 3,000.
Result: 389 populated cells of 1,836, 1,384 build regions. It plans only.
Watch for this trap: LINZ WFS returns lon,lat where the spec says
lat,lon, and the wrong order returns a valid response with zero
buildings. The planner first checks a box known to hold thousands.
Data sources are keyed by location. src/surveys.py searches LINZ layer
metadata for sources whose published extents contain the region, selects the
newest matching LiDAR and imagery layers, and caches metadata under ignored
data/sources/linz/. If no LiDAR DSM covers the area, the build warns and uses
the synthetic-surface path rather than silently selecting another region’s
data.
Search addresses are sharded by prefix when large, so search scales past a flat index.
One 16-core spot VM, data already on disk, scaled from Queenstown:
| VM-hours | 1 machine | 20 | 50 | 100 | |
|---|---|---|---|---|---|
| face precompute | 1,461 | 61 d | 3.0 d | 1.2 d | 0.6 d |
| region builds | 1,347 | 56 d | 2.8 d | 1.1 d | 0.6 d |
| total | 2,808 | 117 d | 5.8 d | 2.3 d | 1.2 d |
These times assume ideal parallelism. The planner targets at most 3,000 buildings per region, but geometry complexity, I/O and retries can still produce stragglers.
The download does not divide. Queenstown needed 116 GB from LINZ; nationally that is about 26 TB from one service:
| LINZ sustains | download takes |
|---|---|
| 0.5 Gbit/s | 4.8 days |
| 1 Gbit/s | 2.4 days |
| 2 Gbit/s | 1.2 days |
| 5 Gbit/s | 0.5 days |
More machines do not help and may hit a rate limit. Nobody has measured
what LINZ sustains (tools/measure_fetch_rate.py). Measure it on one city
first.
Half the country’s buildings are in the densest 20 of 389 cells (Auckland, Christchurch, Wellington, Hamilton):
| buildings | compute | download | |
|---|---|---|---|
| densest 20 cells | 1,709,541 (50%) | 1,403 VM-h (1.2 days on 50) | 12.9 TB |
| all 389 cells | 3,413,097 | 2,808 VM-h (2.3 days on 50) | 25.7 TB |
Do the cities one at a time and publish as each lands. Each is roughly a day.
Done: buildings as tiles, heat map as tiles, incremental builds, survey registry, derived regions, region packs, the yield-only layer, the output contract, and per-region emit and combine. The distributed fleet is implemented, but a successful multi-worker rehearsal is not established by the code. Start with a one-city dress rehearsal (national-rehearsal.md, archived plan). Design: scale-architecture.md.
None of this needs geometry work to pause; they touch different files.