Measuring Quality Loss in Geocoding Fallbacks

A fallback chain keeps a geocoding service answering when its primary provider fails. What it does not do is keep the answers as good. A rooftop-accurate match degrades to a street-centroid, then to a postcode centroid, then to a town centroid — each step still returns coordinates, each step is still “successful”, and each step is progressively less usable for the spatial joins downstream. If the only signal you collect is availability, a chain running permanently on its third tier looks identical to one that never degraded at all. This guide covers how to measure the quality a fallback costs you, how to express it as a single tracked number, and how to alert when degradation becomes the steady state. It belongs to fallback chains for spatial API failures under geospatial observability architecture fundamentals.

Four geocoding fallback tiers with their positional accuracy and downstream usability Four tiers descend from left to right. Rooftop matching gives accuracy within about five metres and supports parcel-level joins. Street interpolation gives about forty metres and supports street-segment joins but not parcel ones. Postcode centroid gives several hundred metres and supports only area aggregation. Town centroid gives kilometres and supports nothing beyond a map pin. Each tier is drawn with a widening uncertainty circle, and a note states that all four return coordinates and count as successful responses. All four tiers return coordinates; only one of them supports a parcel join tier 1 · rooftop ± 5 m parcel joins valid tier 2 · street ± 40 m segment joins only tier 3 · postcode ± 400 m area aggregation only tier 4 · town ± 3 km a pin, nothing more Positional uncertainty drawn to scale relative to each other: the tier is the whole quality story.

Problem framing: availability is the wrong metric

A fallback chain converts an availability problem into a quality problem, and the instrumentation usually fails to follow.

The first consequence is that the success rate stops being informative. Once tier four returns a town centroid for anything it cannot resolve, the chain’s success rate is approximately one hundred percent regardless of how badly the primary is performing. Charting it produces a flat line through an outage.

The second is that the degradation is invisible downstream. A consumer receiving coordinates has no way to know they came from a postcode centroid unless the response says so. It will happily run a parcel-level spatial join against a point that is four hundred metres from the address, produce a confident and wrong parcel identifier, and the error surfaces weeks later as a data-quality complaint about a completely different system.

The fix has two halves: record the serving tier on every response as a first-class dimension, and propagate it to the consumer so the answer carries its own accuracy.

Implementation: tier as a dimension, accuracy as a value

Instrument the chain so every response records which tier served it and what positional uncertainty that tier implies.

# geocode_chain.py — every response records its tier and its uncertainty.
from dataclasses import dataclass
from opentelemetry import metrics

meter = metrics.get_meter("gis.geocode")
responses = meter.create_counter("gis.geocode.responses_total")
uncertainty = meter.create_histogram(
    "gis.geocode.uncertainty_meters",
    description="Positional uncertainty implied by the serving tier",
)

# Uncertainty is a property of the tier, not of the individual answer — using
# the tier's nominal radius keeps the metric comparable across providers.
TIERS = [
    ("rooftop",  5.0),
    ("street",   40.0),
    ("postcode", 400.0),
    ("town",     3000.0),
]

@dataclass
class Geocoded:
    lon: float
    lat: float
    tier: str
    uncertainty_m: float

def geocode(query: str, layer: str) -> Geocoded | None:
    for tier, radius in TIERS:
        result = PROVIDERS[tier].lookup(query)
        if result is None:
            continue
        attrs = {"layer": layer, "tier": tier}
        responses.add(1, attrs)
        uncertainty.record(radius, attrs)
        # The tier travels with the answer so consumers can refuse it.
        return Geocoded(result.lon, result.lat, tier, radius)

    responses.add(1, {"layer": layer, "tier": "none"})
    return None

Returning the tier and uncertainty to the caller is the part most implementations omit, and it is the part that prevents the silent-wrong-join failure. A consumer performing a parcel-level assignment can then require uncertainty_m under its own tolerance and skip rather than fabricate — the same discipline the bounded predicate applies in bounding nearest-neighbour joins with a distance limit.

With the tier as a dimension, a single tracked number summarises chain quality: the share of responses served at or above the tier the platform promises.

Q=∑t≤t∗nt∑tntQ = \frac{\sum_{t \le t^{*}} n_t}{\sum_{t} n_t}

where t∗t^{*} is the lowest acceptable tier and ntn_t the responses served at tier tt. Tracking QQ rather than availability makes an outage visible even while every request succeeds.

Alerting on degradation as a steady state

Two rules, catching the sudden case and the chronic one.

groups:
  - name: geocode-quality
    rules:
      # Sudden: acceptable-tier share drops — the primary is failing now.
      - alert: GeocodeQualityDrop
        expr: |
          sum by (layer) (rate(gis_geocode_responses_total{tier=~"rooftop|street"}[15m]))
            / clamp_min(sum by (layer) (rate(gis_geocode_responses_total[15m])), 1)
          < 0.95
        for: 15m
        labels: { severity: critical, data_domain: spatial }
        annotations:
          summary: >-
            Only {{ $value | humanizePercentage }} of geocodes for
            {{ $labels.layer }} served at an acceptable tier

      # Chronic: the chain has been degraded for a day and nobody noticed,
      # because availability never moved.
      - alert: GeocodeChronicDegradation
        expr: |
          avg_over_time(
            (
              sum by (layer) (rate(gis_geocode_responses_total{tier="rooftop"}[1h]))
                / clamp_min(sum by (layer) (rate(gis_geocode_responses_total[1h])), 1)
            )[24h:1h]
          ) < 0.80
        for: 2h
        labels: { severity: warning, data_domain: spatial }

The chronic rule matters more than it looks. Fallback chains are designed to hide failure, and they succeed: a provider that quietly stopped returning rooftop matches after a contract change can serve street centroids for weeks without a single availability alert. The 24-hour average is what surfaces it.

Availability and tier-quality plotted together through a provider degradation Two series are plotted over the same three days. The availability series stays flat at essentially one hundred percent throughout. The acceptable-tier share series sits near ninety-eight percent for the first day, then steps down to about sixty percent when the primary provider begins failing, and stays there. A marker shows that no availability-based alert would fire at any point, while a tier-quality alert fires at the step. A caption states that the chain is working exactly as designed and that is the problem. The chain is working exactly as designed — which is why availability never moves 100% 60% 20% availability — flat, uninformative acceptable-tier share — the real signal primary starts failing tier-quality alert fires; availability alert never does day 0 day 1 day 2 day 3 Chronic degradation is the normal outcome of an unmonitored fallback chain, not an unusual one. Stored corpus tier mix accumulating low-quality records over successive degradations A stacked bar per month shows the tier composition of the stored geocoded corpus. In early months rooftop matches dominate. Two degradation episodes add visible bands of street and postcode matches that persist in later months because nothing re-geocodes them. By the final month a fifth of the corpus is below rooftop quality. A note states that the live response mix recovered fully after each episode while the stored mix did not. The live mix recovers; the stored corpus does not JanAprAug Green rooftop, amber street, rose postcode. Nothing removes the amber and rose bands but a re-geocoding pass.

Verification

Force each tier in turn by disabling the tiers above it and confirm the counter attributes responses to the correct tier and the histogram records the matching uncertainty. A chain whose tier labels are assigned optimistically — reporting rooftop for a result that actually came from the street interpolator — produces a quality metric that is worse than none, because it is confidently wrong.

Then confirm the tier reaches the consumer. Call the service through its real interface and assert that the response payload carries the tier and uncertainty fields. An internal metric that never leaves the service cannot prevent a downstream parcel join from using a postcode centroid.

Gotchas

Uncertainty derived from the provider’s own confidence score. Provider confidence scores are not comparable across providers and are frequently optimistic. Using the tier’s nominal radius keeps the metric stable when providers change.

Treating tier="none" as a failure only. A no-match is a legitimate and often preferable outcome. Count it separately from a low-tier match, because a chain tuned to always answer will trade honest non-matches for bad coordinates.

Alerting only on the sudden drop. The chronic case is the common one and needs its own longer-window rule.

No per-layer dimension. Different consumers need different minimum tiers; an address-validation flow may accept postcode centroids while a parcel assignment must not. Carrying layer lets one chain serve both with different thresholds.

FAQ

Should the chain refuse to fall back below a tier?

Make it configurable per caller rather than global. A caller that only needs a map pin is well served by a town centroid; a caller performing a parcel join is actively harmed by it. Passing a minimum acceptable tier with the request, and returning no match below it, is the cleanest expression of that.

How does this relate to circuit breakers?

Circuit breakers decide when to stop trying a failing provider; tier quality measures what it costs when they do. Both are needed, and the tuning of the former is covered in tuning circuit breakers for geocoding fallbacks.

Can the uncertainty be stored with the geocoded feature?

It should be. Persisting the tier and uncertainty alongside the coordinates means any later analysis can filter by accuracy, and a re-geocoding pass can target only the low-tier records rather than reprocessing everything.

How should batch re-geocoding use these signals?

Treat the stored tier as a work queue. A nightly job that re-geocodes only records currently held at tier three or below, and only where the primary provider is healthy, recovers accuracy without reprocessing the whole corpus. Track the tier distribution of the stored corpus as its own gauge alongside the live response distribution: the two answer different questions, since the live one tells you how the chain is behaving right now and the stored one tells you how much low-quality data you are still carrying from past degradations.

What if a provider’s tiers do not map cleanly onto ours?

Define the platform’s tiers by usable accuracy rather than by provider vocabulary, and map each provider’s result types onto them. The mapping is a small table that belongs in the same registry as the rest of the layer configuration, and reviewing it when a provider changes is far easier than reasoning about a dozen provider-specific labels.

Should the tier influence retry behaviour?

It should influence whether a retry is worth attempting at all. A request that resolved at tier one needs no retry; one that fell to tier three is a candidate for a later re-attempt once the primary recovers, and one that returned no match at all usually is not, because the address is genuinely unresolvable rather than temporarily unavailable. Making that distinction from the tier rather than from the response status is what keeps a recovery queue from filling with requests that will never succeed.