Corridor LiDAR processing pipeline
Corridor LiDAR processing for a state road authority: mobile-mapping scans in, per-segment certified DTM deliveries out, roughly 200 km of highway a week from one operator seat.
The situation.
State road authority, 4,100 km of sealed corridors resurveyed on a rolling two-year cycle.
ScopeEnd-to-end pipeline for linear-corridor mobile-mapping scans: RAW trajectory smoothing against a CORS network, scan-to-trajectory registration, ground/non-ground classification, breakline-enforced DTM generation, and delivery packages conforming to the authority GIS schema. Acceptance criteria were positional accuracy per 100 m segment, not per dataset.
Target throughput of ~200 km of corridor scans per week with a single operator, accuracy certificate issued per delivery batch.
The system is a staged pipeline, and each stage owns one claim about the data. A mobile-mapping rig writes raw lidar streams plus GNSS and inertial observations to S3 as tiles. The first stage smooths the raw trajectory against a CORS network, which anchors every scan position in an absolute frame. Registration binds each scan to that trajectory, per corridor and per epoch. Classification separates ground from non-ground returns, breaklines get enforced where the terrain model would otherwise lie through culverts and kerbs, and the DTM is built from the classified points. PostGIS holds the corridor model, the 100 m segments and the acceptance ledger, one row of computed residuals per segment. The last stage emits delivery packages shaped to the authority's GIS schema, each carrying its own per-segment accuracy certificate. Python orchestrates the stages and hosts the certification logic; C++ carries the heavy numerics.
Considered: Accept each corridor as one dataset and certify positional accuracy in aggregate.
Chose: Certify per 100 m segment, matching the authority's acceptance model.
Because: Averages hide weak blocks. Segment-level certificates are also what the client acts on, since maintenance budgets move per segment.
Considered: Cut fresh ground control every epoch, bridge decks included.
Chose: Constrain the trajectory with control points extracted from the previous epoch and blend trajectories with constrained splines.
Because: On a two-year cycle, re-surveying control per deck is an endless cost. The previous epoch is a control net the authority has already accepted.
Considered: One general classifier for every land-cover class.
Chose: A second classifier pass keyed to local height variance and width of return, applied where riparian segments fail the first pass.
Because: Sparse understorey and low vegetation sit at the same height. Return geometry separates them; height alone never will.
Considered: A single-language Python pipeline for simplicity of staffing.
Chose: A C++ numerics core with Python orchestration and certification.
Because: A 4,100 km programme moving about 200 km a week punishes slow inner loops, and that throughput was not negotiable.
PDAL earns its place by making the stages explicit: every filter and writer is a named step, re-runnable on a single tile when a result looks wrong. PostGIS is the spine because the authority's schema is relational and segment-shaped, and the acceptance ledger wants the same transactional guarantees as any database. S3 keeps raw tiles and delivered packages on one object tier, so a batch stays reproducible from original data years later. C++ exists for the registration and spline work where per-point cost dominates. Python wraps the stages and runs the certification checks.
The numbers that matter here came from acceptance testing. Before the fix, registration drift on bridge decks landed at 6 to 11 centimetres wherever GNSS multipath peaked; after previous-epoch constraints and constrained-spline blending, drift held under 3 cm, inside the tolerance the per-100 m certification checks against. Adding a geometry-keyed second classifier pass cut manual correction area by 71 percent across the programme. Steady-state throughput settled at about 200 km of corridor scans per week with a single operator. The acceptance criterion is positional accuracy per 100 m segment, not per dataset, and the verified outcome is an accuracy certificate issued per delivery batch: every batch leaves the pipeline with segment-level residuals attached and a certificate to match.
What discovery found
The client had the acceptance model before it had the pipeline: positional accuracy certified per 100 m segment across 4,100 km of sealed corridor resurveyed on a rolling two-year cycle. That sentence shaped the whole build. Most of the value sits in per-segment evidence a certificate can point atand the pipeline delivers exactly that. The corridor in question runs the Hume Highway between New South Wales, the ACT and Victoria. Epcotec had certified surveys per 100 m segment against GNSS control since 2011 on Alberta roads, so the ideas were old. Scaling them to a rolling programme with one operator seat was the new part.
The hard problem
Two failure modes fought the pipeline, and both were quiet. On bridge decks, GNSS multipath peaked and registration drift landed at 6 to 11 centimetres, far outside tolerance. On riparian segments, the classifier confused sparse understorey with low vegetation, and the manual correction queue grew faster than the line could absorb it. A certificate computed from a wrong trajectory still produces plausible numbers, which is why per-segment residuals are a first-class output: drift only becomes visible when the evidence is granular enough to show it.
The build
The deck fix stops trusting each epoch in isolation. Control points extracted from the previous epoch become constraints, and trajectory blending runs as constrained splines, with the constraints doing the smoothing, which holds drift under 3 cm where the GNSS solution degrades. The classifier gained a second pass keyed to geometry: local height variance and width of return, the features that tell a thin sapling from a grass mat at identical height. Manual correction area fell by 71 percent. Neither change is exotic. Both only work because epoch joins and per-segment residuals stay visible in PostGIS from classification through delivery.
Verification
Every segment carries its own residual record, checked by code before anything ships. A failing segment surfaces before packaging, because the certificate is computed from the ledger rather than asserted at the end. Dataset-average accuracy stays out of the delivery on purpose: the acceptance model is per segment, so the certificate is per segment.
Handover
Delivery packages conform to the authority's GIS schema directly, so nothing gets reshaped on the client side. Operators read stage status and the per-batch log; the two failure modes that dominated discovery, bridge-deck drift and riparian misclassification, now arrive as ledger numbers during the run. The two-year cycle runs without us.