Prepare, build, review and release a district dataset. Run commands from the
project folder with .venv active. Prerequisites:
local setup, team access.
District builds run on the build VM, never on a laptop. A laptop builds one small region at a time (see the quickstart). For a shared-VM developer preview rather than a release, use the cloud quickstart.
flowchart TB
Config["Choose regions in config.py"] --> Fetch["Fetch missing regional inputs and wide DEM"]
Fetch --> Ownership["region_build.py: prepare outline ownership"]
Ownership --> Predict["Optional selected-face precompute: predict_faces.py"]
Predict --> District["run_district_build.sh on build VM"]
District --> Served[("Per-region outputs + combined served set")]
Served --> Review["Inspect outputs and run predeploy_check.py"]
Review --> Gate{"Gate passes or production override reviewed?"}
Gate -->|no| Fix["Correct inputs or code before rebuilding"]
Fix --> District
Gate -->|yes| Publish["publish_served.py: versioned tiles bucket"]
Publish --> Switch["Update site-config.js release pointer"]
Arrows show the operational order, not that every command invokes the next.
Input acquisition, ownership and face prediction are preparation outside the
district script. The remote launcher can fetch with --fetch, but does not
run face prediction. For stage order and differences between launchers, use
the execution reference.
Regional DSM, DEM, LiDAR-index and aerial-photo layers are discovered from
LINZ metadata by src/surveys.py and cached locally under ignored
data/sources/linz/. Estimates pin the selected layer IDs for the job.
| Data | Source | Used for |
|---|---|---|
| Building outlines, layer 101290 | LINZ Data Service (WFS 1.0.0) | The buildings estimated. src/fetch_data.py, src/fetch_regions.py |
| Regional LiDAR 1 m DSM and bare-earth DEM | LINZ Exports API; covering layers are selected from LINZ metadata | Roof and obstacle heights, shading and heat map |
| LiDAR tile index and raw point cloud | LINZ metadata and a verified public bulk source when available | Point-level roof evidence; otherwise the build uses its DSM without stale tile manifests |
| Regional aerial imagery and survey-year reference imagery | LINZ metadata; coverage and year are checked against the selected area and DSM | Roof lines, obstructions and photo alignment. Imagery is optional |
| NZ 8 m DEM, layer 51768 | LINZ | Distant horizons. Fetched to data/dem_wide_mosaic.tif by src/fetch_dem_wide.py |
| NZ Addresses, layer 123113 | LINZ | Address labels and search. src/add_addresses.py |
| Measured sunlight | NIWA SolarView GHI (Wakatipu basin); else the nearest NIWA measured-radiation station | Sunlight calibration. src/solar_model.py |
| ERA5 daily radiation, 1940–2025 | Copernicus, via Open-Meteo (CC BY 4.0) | Scales calibration between places; low-solar runs. climate/era5/, tools/pack_era5_daily.py |
| District plan zones | Queenstown Lakes District Council | Home or business pricing. tools/fetch_zoning.py |
| Aerial basemap | LINZ Basemaps | Map display; reference photo for lean. rooftop-solar.html, src/register_imagery.py |
| World imagery basemap | Esri World Imagery | Low-zoom map display only |
Where there is no LiDAR (Cromwell, Glenorchy), the survey is marked
"elevation": "synthetic" and the build uses a modelled surface
(src/synthesize_dsm.py).
Check LINZ has outlines, DSM and imagery for it, then add the region and its
bounding box to config.REGIONS. The original pilot is pilot; its inputs
start in data/ and are linked into data/regions/pilot/.
.venv/bin/python src/fetch_regions.py frankton_flats # one region
.venv/bin/python src/fetch_regions.py # every region
.venv/bin/python src/fetch_dem_wide.py # wide DEM, if missing or too small
Each region lands in data/regions/<region>/. Fetching resumes: existing
files are skipped. Imagery is fetched in chunks of at most 8 km². Areas with
no imagery log a warning and build from LiDAR alone. src/fetch_data.py
fetches the legacy pilot inputs into data/.
After fetching every region you will combine, give each building one owning region:
.venv/bin/python src/region_build.py
Prepare selected-face readings with tools/predict_faces.py --region REGION
when using the selected-faces geometry path. The district script consumes
existing readings; it does not run that precompute itself. Before a geometry
build, authorize demolition screening, including
external API cost and image transfer. Then, on the VM:
bash src/run_district_build.sh # incremental: stale buildings only
bash src/run_district_build.sh --regions "pilot frankton_flats"
bash src/run_district_build.sh --force # rebuild everything
bash src/run_district_build.sh --yield-only # solar model changed: recompute kWh, no LiDAR
Check data/build_logs/<region>.log and pre-plan warnings. Outline fitting
and repeated-roof consensus can fail without stopping the build; they run
before normal/forced builds, not yield-only builds. A failed region stops
the final combine.
--regions restricts the build loop, not the final combined map. Combine
includes available summaries under data/out/, excluding
config.SERVED_EXCLUDE; unchanged regions can be included.
The execution reference explains stage order, sidecars and marker freshness. Builds are incremental from per-building keys; local pipeline and worker reruns have different reuse rules.
For fast layout work on one area: bash src/run_dev_loop.sh pilot.
Every engine runs the demolished stage. Its free screen runs every time;
its referee, which sends survey/current image crops to the Anthropic API,
is off unless approved at the start of the run. Each entry point asks a
yes/no question with a conservative estimate (USD $0.004 a roof, stopping
near SOLAR_DEMOLISHED_MAX_USD, default USD $5 a region; not a spending cap)
and approves only on an explicit yes. Enter, any other answer and
non-interactive runs decline. The quickstart’s VM/preview estimate does not
include this cost.
Non-interactive approval: --paid-referee on tools/pipeline.py and
tools/remote_build.sh, the paid_referee input on the GitHub workflow, or
SOLAR_DEMOLISHED_REFEREE=1 in the environment of tools/fleet.sh up or a
direct engine call. --no-paid-referee declines without asking, as does
--yes on tools/pipeline.py (it approves the VM cost estimate only). The
execution reference
lists where each entry point asks.
SOLAR_DEMOLISHED=0 disables the stage; with the referee declined, cached
verdicts still apply. Yield-only builds skip screening.
Missing imagery/key, API failures or exhausted usage can leave only cached verdicts applied; a zero exit code does not prove every roof was screened. The stage implementation defines the cache, hand-drawn exemptions and withholding criteria.
src/audit_layout_quality.py: low fill, scattered arrays, mixed angles,
too much obstruction.src/render_top_movers.py: picture cards of the buildings that changed
most.src/render_building_debug.py <building_id>: one roof’s faces, planes and
panels.src/validate_obstructions.py when obstruction rules change.tools/layout_ab.py, tools/layout_health.py, tools/face_area_check.py).When code changed, run the matching tests before pushing; the reviewer’s guide lists the fast suite, synthetic build and geometry checks.
.venv/bin/python tools/pipeline_serve.py --root . --port 8000
Open http://localhost:8000/rooftop-solar.html. This loopback server supports the
byte-range requests PMTiles needs. Stop it with Ctrl+C.
The live refit server is a separate developer tool requiring root-level pilot inputs, not the regional preview path; see execution boundaries.
The site (rooftop-solar.html, the Build data page and their scripts) and
rendered documentation are deployed by the
static Pages workflow. The data is served
from the Google Cloud tiles bucket, one folder
per release: gs://solar-potential-map-tiles/v<N>/. site-config.js
(dataBase, dataVersion) tells the page which folder to read.
From GitHub (Actions > “Build and publish the map”) or a laptop
(tools/remote_build.sh --publish), the VM builds, gates and publishes. To
review a VM build on a laptop first:
tools/deploy_from_vm.sh # pull the served set, run the gate, stop
tools/deploy_from_vm.sh --push # ...and publish if the gate passes
tools/deploy_from_vm.sh --push-reviewed "why" # publish past a gate whose list you have checked
The gate is tools/predeploy_check.py: it compares the new build against the
live one. Publishing (tools/publish_served.py) uploads a new version
folder, keeps the newest three, and commits only the site-config.js change.
A version folder is never changed after upload.
Roll back by pointing site-config.js dataBase and dataVersion at an
older folder that is still in the bucket.