Monitoring Topology for Multi-Region GIS

When spatial pipelines span more than one geographic region, a flat monitoring setup that scrapes every transformation worker from a single central backend collapses under cross-region latency, replication skew, and data-residency rules. This page is for the data engineers, GIS platform admins, and SREs who run vector ETL, tile generation, and PostGIS workloads across regions and need a telemetry layout that stays high-fidelity during partial network partitions without leaking coordinate metadata across jurisdictions. It sits under Geospatial Observability Architecture & Fundamentals and assumes you have already settled the canonical signal names from the Geospatial Metric Taxonomy for ETL; here we focus on where those signals are collected, aggregated, and routed once your deployment crosses a regional boundary.

Architecture

A resilient monitoring topology for multi-region GIS is hierarchical: regional edge collectors ingest spatial ETL telemetry close to where geometry is processed, pre-aggregate and filter it, then forward a reduced signal set to a global control plane. This two-tier shape prevents cross-region network saturation and stops a localized failure — a stuck CREATE INDEX CONCURRENTLY, a reprojection worker thread-pool stall — from cascading into a global alert storm. Each regional node behaves as an independent observability domain that captures pipeline execution traces, spatial index rebuild durations, and coordinate reference system (CRS) transformation metrics even when the link back to the control plane is down.

The placement of these tiers is not purely a performance decision. Telemetry routing has to respect Defining Spatial Data Trust Boundaries so that cross-region aggregation never violates jurisdictional data-residency policy or exports sensitive coordinate metadata into an unauthorized processing zone. In practice that means the edge collector — not the global backend — is the egress control point: it strips or hashes high-precision coordinate attributes, enforces label allowlists, and emits only the dimensional rollups the control plane is permitted to see. Which signals are even eligible for collection at each tier is governed by the Observability Scoping Rules for Vector Data, because point, line, and polygon pipelines carry different baselines and cardinality budgets.

The topology must explicitly map three logical zones onto collectors: ingestion (raw WKB/GeoJSON arrival and SRID validation), transformation (CRS reprojection, topology validation, spatial joins), and publishing (tile generation and serving). Isolating telemetry at the regional edge keeps signal capture high-fidelity during partial network partitions, and it keeps compliance-relevant logs from ever traversing a region they are not allowed to enter. The instrumentation that feeds these collectors is the same SDK and collector wiring covered in OpenTelemetry Integration for GIS Pipelines; this page assumes those spans and metrics already exist and concerns itself with their regional routing.

A regional collector built on the OpenTelemetry contrib distribution filters the GIS namespace before anything leaves the region:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 10s
    send_batch_size: 1000
  filter:
    metrics:
      include:
        match_type: regexp
        metric_names:
          - "gis\\.etl\\..*"
          - "gis\\.spatial\\..*"
  # Egress control: drop high-precision coordinate attributes before forwarding.
  attributes/residency:
    actions:
      - key: gis.feature.centroid
        action: delete
      - key: gis.region
        action: upsert
        value: "region-a"

exporters:
  prometheus:
    endpoint: 0.0.0.0:8889
    namespace: gis_regional
  otlphttp/global:
    endpoint: https://global-control-plane.internal:4318
    compression: gzip

service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [filter, attributes/residency, batch]
      exporters: [prometheus, otlphttp/global]

Metric Specification

Standard infrastructure telemetry is insufficient for geospatial workloads: it measures CPU and queue depth but says nothing about geometry validity, CRS drift, or tile-publish lag. The topology layer adds region-aware dimensions to every spatial signal so that an SRE can answer “which region, which CRS, which dataset” without re-deriving it during an incident. Every metric below is tagged with a region identifier, a CRS code, and a data-lineage marker.

Per-metric thresholds are necessary but not sufficient; a control plane also needs one rolled-up number per region so dashboards can rank regions by health at a glance. Define a regional telemetry fidelity score as a weighted blend of signal completeness, latency headroom, and drop rate:

$$F_{region} = w_c \cdot \frac{C_{received}}{C_{expected}} \;+\; w_l \cdot \left(1 - \frac{L_{p95}}{L_{budget}}\right) \;+\; w_v \cdot \left(1 - r_{drop}\right)$$

where $C_{received}/C_{expected}$ is the fraction of expected spatial datapoints that actually reached the control plane, $L_{p95}/L_{budget}$ is the p95 forwarding latency against its budget, $r_{drop}$ is the edge-collector drop ratio from gis.collector.export_dropped_total, and the weights satisfy $w_c + w_l + w_v = 1$. A region whose collector is silently dropping spatial metrics under backpressure will see $F_{region}$ fall well before any individual per-metric alert fires, which makes it the right top-level signal to page on.

Pipeline Integration & Configuration

Integrating spatial telemetry means instrumenting both the transformation layer and the spatial database engine, then attaching region and CRS context to the spans the collector forwards. The OpenTelemetry SDK attaches custom attributes to the spans that represent geometry processing steps, which is what makes a single feature traceable from ingestion through reprojection to tile publication across regions:

import time
from opentelemetry import trace, metrics
from opentelemetry.trace import Status, StatusCode

tracer = trace.get_tracer("gis.etl.spatial_transform")
meter = metrics.get_meter("gis.etl.spatial_transform")

join_latency = meter.create_histogram("gis.spatial.join.latency_ms", unit="ms")

def process_spatial_join(features_a, features_b, region, algorithm="rtree"):
    with tracer.start_as_current_span("spatial_join_execution") as span:
        # region + algorithm are required dimensions so the control plane
        # can attribute latency to the correct edge collector.
        span.set_attribute("gis.region", region)
        span.set_attribute("gis.algorithm", algorithm)
        span.set_attribute("gis.input_count_a", len(features_a))
        span.set_attribute("gis.input_count_b", len(features_b))

        start_time = time.perf_counter()
        try:
            result = execute_join(features_a, features_b, algorithm)
            span.set_status(Status(StatusCode.OK))
            return result
        except Exception as e:
            span.set_status(Status(StatusCode.ERROR, str(e)))
            span.record_exception(e)
            raise
        finally:
            duration_ms = (time.perf_counter() - start_time) * 1000
            join_latency.record(duration_ms, {"algorithm": algorithm, "region": region})

On the database side, expose index and replication health directly from PostGIS so the collector can scrape it rather than inferring it from query latency. A view like the following surfaces fragmentation and feature counts per spatial table, which the regional collector turns into gis.index.fragmentation_pct and lineage markers:

-- Per-table spatial health snapshot scraped by the regional collector.
SELECT
    c.relname                              AS table_name,
    ST_SRID(t.geom)                        AS srid,
    COUNT(*)                               AS feature_count,
    SUM(CASE WHEN NOT ST_IsValid(t.geom) THEN 1 ELSE 0 END) AS invalid_geoms,
    ST_Extent(t.geom)                      AS regional_extent
FROM spatial_features t
JOIN pg_class c ON c.relname = 'spatial_features'
GROUP BY c.relname, ST_SRID(t.geom);

When the source feed for a region passes through cross-region replication, align its freshness budget with the Tracking Spatial Data Freshness SLAs baselines so that gis.region.replication_lag_s is interpreted against the same SLA the rest of the platform uses.

Threshold Design & Alerting Logic

Spatial workloads degrade non-linearly — a GiST index a few percent past its fragmentation knee can double planner cost — so thresholds must reflect geometry complexity, index health, and per-region data volume rather than flat infrastructure limits. The rules below tier severity by who must act and how fast.

groups:
  - name: gis_spatial_reliability
    rules:
      # WARNING — degraded but serving; data-engineering reviews within the hour.
      - alert: SpatialJoinLatencyDegradation
        expr: |
          histogram_quantile(0.95,
            rate(gis_spatial_join_latency_ms_bucket[5m])) > 2000
        for: 3m
        labels:
          severity: warning
          team: data-engineering
        annotations:
          summary: "95th percentile spatial join latency exceeds 2s in {{ $labels.region }}"
          description: "Region {{ $labels.region }} has degraded spatial join performance. Check index fragmentation and CRS transformation overhead."

      # CRITICAL — query planner efficiency at risk; db-admin acts now.
      - alert: IndexFragmentationCritical
        expr: gis_index_fragmentation_pct > 15
        for: 5m
        labels:
          severity: critical
          team: db-admin
        annotations:
          summary: "GiST index fragmentation exceeds safe threshold"
          description: "Table {{ $labels.table_name }} in {{ $labels.region }} requires REINDEX. Fragmentation at {{ $value }}% degrades planner choices."

      # WARNING — upstream data quality; compliance-ops verifies source contract.
      - alert: GeometryValidationFailureSpike
        expr: rate(gis_etl_geom_validation_failures_total[10m]) > 50
        for: 5m
        labels:
          severity: warning
          team: compliance-ops
        annotations:
          summary: "High rate of invalid geometries in {{ $labels.region }} ingestion"
          description: "Source data quality degradation detected. Verify schema contracts and coordinate precision for {{ $labels.source_system }}."

      # DYNAMIC_BASELINE — collector silently dropping spatial metrics under backpressure.
      - alert: RegionalTelemetryFidelityDrop
        expr: |
          (rate(gis_collector_export_dropped_total[10m]) /
           clamp_min(rate(gis_collector_export_received_total[10m]), 1)) > 0.02
        for: 5m
        labels:
          severity: critical
          team: observability-platform
        annotations:
          summary: "Edge collector dropping spatial metrics in {{ $labels.region }}"
          description: "Drop ratio above 2% means dashboards for {{ $labels.region }} are blind. Increase batch.send_batch_size and enable disk buffering."

The fidelity-drop rule deliberately watches the collector itself: in a multi-region topology the most dangerous failure is not a loud spike but a quiet blind spot where a partitioned region stops reporting and every per-metric alert goes green by omission.

Failure Modes & Edge Cases

• Silent blind region. A network partition severs an edge collector from the control plane. Per-metric alerts evaluate to “no data” and most alerting backends treat that as healthy. Diagnose by alerting on absence — a RegionalTelemetryFidelityDrop and an absent_over_time(gis_region_replication_lag_s[10m]) rule per region — rather than only on thresholds being exceeded.
• CRS mismatch masking latency. When a region reprojects through an unexpected src_crs, gis.crs.transform.duration_ms rises but is averaged away across the global histogram, so the p95 looks fine. Diagnose by always querying the histogram with the region and src_crs dimensions intact; a global rollup hides single-region drift. This is the same drift the Coordinate Reference System Validation checks catch at ingestion.
• Replication lag misread as freshness. Cross-region replication delay inflates feature staleness in the downstream region even though the source pipeline is healthy. Diagnose by separating gis.region.replication_lag_s from source freshness; if replication lag accounts for the gap, the fix is replication tuning, not the ETL job.
• Collector backpressure dropping topology spans. Under load the batch processor drops datapoints, and because topology-error spans are relatively rare they are disproportionately lost. Diagnose by comparing gis.collector.export_dropped_total against gis.etl.geom_validation_failures_total; a rising drop counter with a flat failure counter means you are losing exactly the signals you most need.
• Alert storm from a shared upstream. A single bad source feed fans out to every region simultaneously, firing identical alerts N times. Diagnose by grouping alerts on source_system rather than region, so a common-cause upstream failure pages once with the affected regions listed.

Troubleshooting Checklist

1. Identify the lag source. Query pg_stat_activity in the affected region for long-running CREATE INDEX CONCURRENTLY or VACUUM FULL on spatial tables before assuming the collector is at fault.
2. Confirm the region is actually reporting. Check gis.collector.export_dropped_total and the RegionalTelemetryFidelityDrop alert; a blind region produces false-healthy dashboards.
3. Validate trace propagation. Ensure W3C trace context survives the message queue (Kafka/RabbitMQ) carrying spatial payloads, so a feature stays traceable across the region boundary.
4. Inspect CRS transformation tails. Watch the gis.crs.transform.duration_ms p99 with region and src_crs kept; high tails indicate thread-pool exhaustion in the reprojection worker, not a global slowdown.
5. Audit egress trust boundaries. Verify the edge collector’s residency processor strips coordinate metadata before forwarding. Review the egress configuration against Best practices for defining trust boundaries in PostGIS so data-residency constraints are enforced at the collector, not the backend.
6. Mitigate collector backpressure. If a region drops spatial metrics, raise batch.send_batch_size and enable a local disk buffer (file_storage extension) so the collector survives transient partitions without losing topology spans.
7. Engage graceful degradation. If spatial APIs or transformation services are partially down, fall back rather than corrupt downstream tiles — configure Fallback Chains for Spatial API Failures so a region routes to bounding-box pre-filtering or a simplified-geometry cache and emits gis.pipeline.circuit_open instead of publishing bad data.

Maintaining strict per-region dimensional tagging, alerting on signal absence as well as threshold breaches, and aligning thresholds with spatial-workload non-linearity is what lets a platform team keep deterministic observability across a multi-region GIS deployment.

Related Pages

• Geospatial Observability Architecture & Fundamentals — the parent overview tying topology, trust boundaries, and instrumentation together.
• Defining Spatial Data Trust Boundaries — where coordinate transformations and residency rules are drawn, which the edge collector enforces.
• OpenTelemetry Integration for GIS Pipelines — the SDK and collector wiring that produces the spans this topology routes.
• Fallback Chains for Spatial API Failures — graceful degradation paths a region takes when spatial services breach SLOs.
• Best practices for defining trust boundaries in PostGIS — collector-egress residency enforcement for cross-region telemetry.