Geospatial Deep Learning

Purpose: deep learning on Earth observation with the two failure modes that dominate this field designed out from the start: spatial leakage (inflated metrics from nearby train/test pixels) and georeferencing loss (predictions that no longer align with the map).

Characterise the label set before naming an architecture

Architecture advice given without knowing the label set is guesswork. Before recommending U-Net versus a foundation model versus a non-deep baseline, state or ask for:

• Label count and labelled area — polygons alone say nothing; 40 polygons covering 2 ha and 40 covering 2 000 km² are different problems.
• Geographic spread — are the labels clustered in one scene, one season and one sensor, or distributed across the deployment domain? Clustered labels cap what any model can generalise to, and they decide whether a geographically independent validation split is even constructible.
• Class balance and minority-class pixel fraction, so loss and sampling choices are grounded rather than assumed.
• Deployment geography — where predictions will be made, relative to where the labels are.

Do not answer "fine-tune a large model or use a simpler approach" before these are known. When the user has not supplied them, ask and give the provisional recommendation conditioned on the answers ("if the 40 polygons sit in one scene, then …; if they span the region, then …"), never a single unconditional recommendation.

Problem framing first

Before any deep model: run a cheap baseline (random forest on bands+indices, or thresholded index). If the DL model can't beat it clearly, the problem is data, not architecture. segmentation-models-pytorch and torchgeo cover most needs — don't hand-build architectures without a reason.

Chipping (dataset construction)

• Chip size: 256–512 px; stride < chip size only for training (overlap augments), never let overlapping chips straddle the train/val boundary.
• Preserve georeferencing: store each chip's transform/bounds (torchgeo datasets or a sidecar index in GeoParquet). A prediction you can't put back on the map is worthless.
• Keep chips in the native data range; normalize with dataset-computed per-band statistics (ImageNet stats only for 3-band RGB with a pretrained encoder, and say so).
• Class imbalance is the norm (buildings ≈ 2-5% of pixels). Log per-chip class fractions; oversample positive-containing chips rather than distorting the loss beyond recognition.

Split policy — the non-negotiable

Split by geographic block or scene, never by random chip. Adjacent chips are near-duplicates; random splits produce beautiful, fake validation curves. Follow the canonical protocol: ml-experiment-standards → references/spatial-cv-protocol.md. For generalization claims across regions, hold out an entire region.

Training defaults

• Loss: Dice + CE (segmentation, imbalanced); plain CE when balanced; Focal only after comparing — it's not a free win.
• Augmentation: flips/rot90 are safe for nadir imagery; be careful with color jitter on multispectral (it breaks radiometric meaning — prefer band dropout or slight scaling); never augment in ways that violate the physics.
• Encoder pretrained; multispectral input → inflate/replace first conv, or use an EO foundation model checkpoint (Prithvi, SatMAE, Clay) when bands match.
• Early stopping on val mIoU (patience 10-15); cosine or plateau LR schedule; AMP on by default.
• Log config + metrics + git hash per run — see ml-experiment-standards.

Inference on large scenes

Sliding window with overlap (25-50%) and blending (feather/gaussian or center-crop stitching) to kill tile-edge artifacts. Then:

import rasterio

with rasterio.open(scene_path) as src:
    profile = src.profile
profile.update(count=1, dtype="uint8", nodata=255, compress="deflate")
with rasterio.open(out_path, "w", **profile) as dst:
    dst.write(mask.astype("uint8"), 1)  # same transform/CRS as the scene

Post-process: sieve tiny blobs (min mapping unit), optionally regularize building polygons, and vectorize (rasterio.features.shapes) for GIS delivery. Report metrics AFTER post-processing too — that's what the user ships.

Verification protocol

1. Metrics table: per-class IoU/F1 with CI across seeds or folds.
2. Error map: prediction vs reference overlaid on imagery for 3+ representative areas including a known-hard one.
3. Sanity inference on an out-of-distribution patch (different season/ region) with an honest note on degradation.
4. Alignment check: overlay predictions on the source scene in a GIS at two zoom levels — catches transform bugs instantly.

Pitfalls checklist

• Random chip split → leaked, unreproducible "SOTA".
• Normalizing test data with train-time stats not saved → skewed inference.
• Losing the geotransform in NumPy-land; writing predictions with default north-up transform.
• Tile-edge seams from no-overlap inference.
• uint16 imagery fed to a float pipeline without scaling → dead gradients.
• Accuracy reported on chip level while the product is a stitched map.

Execution contract

• Workflow: frame target and unit of prediction; build chips and labels; create spatial splits; train against a baseline; run overlap-aware inference; validate the stitched product.
• Decision rules: use deep learning only when label volume, spatial texture, compute, and expected uplift justify it; otherwise prefer a simpler remote-sensing or ML workflow.
• Verification protocol: report spatial holdout metrics across seeds or folds, inspect error maps and hard areas, test geographic transfer, and check output georeferencing.
• Failure modes: invalidate results for leaked chips, label misalignment, train/inference normalization drift, tile seams, or metrics computed at the wrong product unit.
• Deliverables: model and configuration, split manifest, preprocessing contract, metrics with uncertainty, georeferenced predictions, error maps, and model card limitations.
• Source freshness: consult the authoritative source registry before selecting framework APIs, datasets, or weights and record the checked date.