Push vs Pull Metric Collection for Spatial Pipelines

A spatial ETL run that reprojects a continent of parcels and exits in forty seconds is invisible to a scraper that arrives on a fifteen-second cadence — it may be caught once, or never, and the metrics it emitted die with the process. This ephemeral-job problem is the sharpest edge of the push-versus-pull decision for spatial pipelines, but it is not the only one: high-cardinality vertex-count histograms, batch jobs with no stable scrape target, and cross-region telemetry aggregation all pull the choice in different directions. This page works through push (OTLP) versus pull (scrape) for exactly these spatial workloads, with configuration and decision guidance. It sits under choosing spatial observability tooling within the broader spatial incident response and observability tooling program.

The two panels are the whole argument. Pull is elegant for the steady state — a scraper walks a fixed set of long-lived targets, and the series count is predictable — but it has a structural blind spot for anything that does not outlive a scrape interval. Push inverts the relationship: the workload initiates delivery, so a job that runs for forty seconds gets its metrics off the box before it dies. Spatial pipelines are unusually rich in exactly the short-lived, bursty, high-cardinality work that stresses this seam, which is why the choice deserves more than a default. It connects directly to the collection patterns in OpenTelemetry integration for GIS pipelines.

The Ephemeral Job Problem

Pull-based collection rests on an assumption that spatial batch work routinely violates: that a metric source is a stable, long-lived endpoint waiting to be scraped. A reprojection job, a nightly tile bake, an ad-hoc ogr2ogr conversion — these spin up, do intense spatial work, and exit. If the scrape interval is fifteen seconds and the job runs for eight, the odds are better than even that no scrape ever coincides with it, and even a captured job reports only a single lifetime snapshot rather than a curve. The metrics that would tell you the job processed 40 million features with a p99 vertex count of 180k simply never reach the backend.

The classic pull-model patch is the Pushgateway: the job pushes its final metrics to a persistent gateway that Prometheus then scrapes. It works, but it comes with sharp edges that bite spatial workloads specifically. Metrics pushed to a gateway persist until explicitly deleted, so a job that pushes a gis_etl_features_processed_total and then never runs again leaves a stale series that looks live forever. And because the gateway coalesces by grouping key, two concurrent reprojection jobs with the same job label overwrite each other. For a fleet of parallel spatial batch jobs, the gateway becomes a correctness hazard, not just an ergonomic one.

Comparison Table

Configuring the Push Path

For an ephemeral spatial batch, push metrics over OTLP with a short export interval and, critically, a forced flush before exit so the last window is never lost. A unique resource attribute per job run keeps concurrent jobs from colliding the way a pushgateway grouping key would:

import uuid
from opentelemetry import metrics
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.exporter.otlp.proto.grpc.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.resources import Resource

def build_meter(job_name: str) -> MeterProvider:
    resource = Resource.create({
        "service.name": job_name,
        "gis.job.run_id": str(uuid.uuid4()),   # unique per run — no cross-job collision
    })
    reader = PeriodicExportingMetricReader(
        OTLPMetricExporter(endpoint="collector:4317", insecure=True),
        export_interval_millis=5000,
    )
    return MeterProvider(resource=resource, metric_readers=[reader])

provider = build_meter("gis.etl.reproject_batch")
meter = provider.get_meter("gis.etl.reproject_batch")
features = meter.create_counter("gis.etl.features_processed_total")
vertices = meter.create_histogram("gis.spatial.vertex_count", unit="vertices")

# ... process features, recording as you go ...
features.add(40_000_000, {"layer": "parcels", "target_crs": "EPSG:3857"})

# Force the final window out before the process exits — the whole point.
provider.force_flush()
provider.shutdown()

The collector absorbs bursts and shields the backend, applying a memory limiter so a flood of high-cardinality vertex histograms during a big bake sheds load gracefully rather than knocking over the TSDB:

# otel-collector-config.yaml (contrib) — absorb bursty spatial batch pushes
processors:
  memory_limiter:
    check_interval: 1s
    limit_percentage: 75
    spike_limit_percentage: 15
  batch/spatial:
    send_batch_size: 2000
    timeout: 5s
service:
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch/spatial]
      exporters: [prometheusremotewrite]

Configuring the Pull Path

Where a workload is long-lived — a tile server, a PostGIS instance behind an exporter — pull is the simpler and more robust choice, and its up metric gives you a liveness signal push cannot match. Scrape it directly and lean on service discovery so new replicas are found without config edits:

scrape_configs:
  - job_name: spatial-tileserver
    scrape_interval: 15s
    kubernetes_sd_configs:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_label_app]
        regex: tileserver
        action: keep
      # Never let a per-feature label become a scrape dimension.
      - source_labels: [feature_id]
        action: labeldrop

If a class of short-lived jobs genuinely must stay on the pull model, route them through a pushgateway but automate cleanup so stale series do not accrete — the job deletes its own group on success:

# Push a batch job's final metrics, then delete the group so it does not go stale.
JOB=reproject_batch; RUN=$(uuidgen)
cat <<EOF | curl --data-binary @- "http://pushgw:9091/metrics/job/${JOB}/run/${RUN}"
gis_etl_features_processed_total{layer="parcels"} 40000000
EOF
# On success, remove the group so the series does not linger forever.
curl -X DELETE "http://pushgw:9091/metrics/job/${JOB}/run/${RUN}"

Decision Guidance

Match the collection model to the workload’s lifetime first. Long-lived services — tile servers, PostGIS behind an exporter, feature APIs — are pull-native: scrape them, take the free up liveness signal, and enjoy the fixed series count. Ephemeral and bursty spatial work — reprojection batches, nightly bakes, ad-hoc conversions — is push-native: give each run a unique resource attribute, force a flush before exit, and let a collector absorb the cardinality with a memory limiter. Resist the temptation to force one model across a mixed fleet; the pushgateway exists precisely to bridge the gap, but its stale-series and grouping hazards mean it should be the exception, wrapped in automated cleanup, not the default. For high-cardinality vertex histograms specifically, push plus collector-side sampling scales better than pull, because you can shed the heavy tail at the edge before it crosses the network — the same edge-aggregation logic behind monitoring topology for multi-region GIS. Whichever model you pick, keep label sets bounded per bounding spatial metric tag cardinality, and remember that this collection fork interacts with — but is separate from — the signal-model fork covered in the OpenTelemetry versus Prometheus-native comparison.

FAQ

Why not just shorten the scrape interval to catch batch jobs?

Because a job that runs for eight seconds still slips between five-second scrapes often enough to be unreliable, and every replica you scrape faster multiplies TSDB write volume across the whole fleet, not just the batch jobs. You would pay a fleet-wide cost to partially fix a batch-only problem. Push captures the job deterministically at a fraction of that cost.

How do I keep pushed metrics from going stale?

Give each job run a unique resource attribute or grouping key and delete the group on completion, as the cleanup example shows. On the pure OTLP push path the problem largely evaporates, because the SDK stops exporting when the process ends and the series naturally ages out of the backend rather than being pinned by a gateway.

Can I lose data if the collector is down when a batch job pushes?

Yes — a push to an unreachable collector can drop unless you enable client-side retry with a bounded queue. For critical batch metrics, configure the OTLP exporter’s retry and a short persistent queue so a brief collector blip does not vaporize the run’s numbers. This is the push analogue of a scraper skipping a tick, but it fails toward loss rather than toward a gap.

Does push break my liveness alerting?

It weakens it. Pull gives you up for free — a target that stops responding is immediately visible. With push there is no target to be down, so you must synthesize liveness from expected-throughput alerts: if gis.etl.features_processed_total stops advancing for a long-lived pusher, that absence is your liveness signal. Design that alert deliberately, because push will not hand it to you.

Related

• Choosing spatial observability tooling — the parent framework that treats collection model as one fork of the tooling decision.
• OpenTelemetry vs Prometheus-native instrumentation for PostGIS — the signal-model fork that interacts with this collection-model choice.
• OpenTelemetry integration for GIS pipelines — the collector patterns the push path builds on.
• Monitoring topology for multi-region GIS — edge-collector aggregation that the high-cardinality push path relies on.