Compliance Score APIs & Payload Contracts

Within the Reporting & Compliance Dashboards pillar, this is the component that exposes stored compliance results as stable, versioned APIs and webhook contracts for ERP/BI/workforce tools. Every stage before it — capture, detection, slot assignment, scoring — exists to produce one durable artifact: the typed compliance payload for a fixture_id at a capture_timestamp. This component’s only job is to hand that artifact to the rest of the enterprise without ever leaking the pipeline’s internals, and to keep doing so unchanged while the pipeline underneath it is rewritten, retuned, and re-versioned. A category manager’s BI dashboard, a workforce-management app that dispatches reset tasks, and an ERP that reconciles on-shelf availability all read from the same contract, so a careless field rename here breaks three downstream systems at once. This page defines the payload as a canonical resource, a runnable typed API layer that serves it, the versioning and rate-limit configuration that keeps it stable, the failure modes that corrupt it in production, and the read-path performance budget you have to design against.

Concept & Data Contract Jump to heading#

The compliance payload is the canonical resource this whole component serves, and every design decision follows from treating it as an immutable, addressable fact rather than a view that can be reshaped per consumer. A payload is uniquely identified by the tuple of store_id, fixture_id, and capture_timestamp; it is written once when the scoring stage finishes and is never mutated in place. Downstream systems address it by that identity, and the API’s contract is a promise that a given identity always resolves to the same bytes with the same field semantics. That immutability is what lets a BI tool cache aggressively, an ERP reconcile deterministically, and an audit replay a score months later against the exact registry_revision that produced it.

The typed fields are the interface. The compliance stage in Planogram Sync & SKU Mapping Strategies emits compliance_percentage as the headline metric, out_of_stock_flags as the set of slots that read vacant, misplaced_sku_list as the foreign SKUs found in the wrong slots, registry_revision as the planogram version the score was computed against, and capture_timestamp as the moment the frame was taken. This component adds nothing to the score — it only stores, indexes, versions, and serves. A stored compliance payload looks like this:

{
  "payload_version": "1.4",
  "store_id": "STR-00417",
  "fixture_id": "BAY-014-SHELF-03",
  "capture_timestamp": "2026-06-28T07:42:11Z",
  "registry_revision": 184,
  "compliance_percentage": 91.3,
  "scored_slot_count": 46,
  "out_of_stock_flags": ["BAY-014-SHELF-03-POS-11"],
  "misplaced_sku_list": [
    { "slot_id": "BAY-014-SHELF-03-POS-08", "expected_sku": "00098765432107", "detected_sku": "00012345678905" }
  ],
  "price_tag_mismatch_count": 2,
  "generated_at": "2026-06-28T07:43:04Z"
}

The JSON Schema that pins this shape is the contract of record, and it is validated on write and on read. The payload_version field is not decorative: it is the discriminator that tells a consumer which schema to expect, and it is the single field that lets this component evolve without breaking anyone. A minimal schema for the resource:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "CompliancePayload",
  "type": "object",
  "required": ["payload_version", "store_id", "fixture_id", "capture_timestamp",
               "registry_revision", "compliance_percentage", "out_of_stock_flags"],
  "properties": {
    "payload_version": { "type": "string", "pattern": "^\\d+\\.\\d+$" },
    "store_id": { "type": "string" },
    "fixture_id": { "type": "string" },
    "capture_timestamp": { "type": "string", "format": "date-time" },
    "registry_revision": { "type": "integer", "minimum": 0 },
    "compliance_percentage": { "type": "number", "minimum": 0, "maximum": 100 },
    "out_of_stock_flags": { "type": "array", "items": { "type": "string" } },
    "misplaced_sku_list": { "type": "array" }
  },
  "additionalProperties": true
}

Note additionalProperties: true — consumers must tolerate unknown fields so that adding a field is a non-breaking change. That single decision, made in the schema, is what separates an additive 1.4 → 1.5 rollout from a breaking 1.x → 2.0 one. The discipline of which changes are additive and which force a major version is owned by Versioning Compliance Payloads Across Store Resets, because a store reset bumps registry_revision and can retire slots mid-history, which is exactly the case that naive versioning gets wrong.

Implementation Architecture Jump to heading#

The API layer is deliberately thin: it validates the request, resolves a query against the payload store, shapes the response through a typed model that is the contract, and returns it. FastAPI with Pydantic response models is the right tool here because the response model is machine-checked against every outbound payload — if a stored record drifts from the declared schema, serialization fails loudly in the service rather than silently shipping a malformed contract to an ERP. Pydantic also generates the OpenAPI document that downstream teams code against, so the contract is published, not described in a wiki.

The response models mirror the resource exactly, and the list endpoint filters by store_id and a date range over capture_timestamp, returns pages via an opaque cursor rather than an offset, and stamps the served payload_version into every item:

from datetime import datetime, date
from typing import Annotated
import base64, json

from fastapi import FastAPI, Query, Depends, HTTPException
from pydantic import BaseModel, Field

app = FastAPI(title="Compliance Score API", version="1.4")


class MisplacedSku(BaseModel):
    slot_id: str
    expected_sku: str
    detected_sku: str


class CompliancePayload(BaseModel):
    """The canonical compliance resource. This model IS the v1 contract."""
    payload_version: str = Field(pattern=r"^\d+\.\d+$")
    store_id: str
    fixture_id: str
    capture_timestamp: datetime
    registry_revision: int = Field(ge=0)
    compliance_percentage: float = Field(ge=0.0, le=100.0)
    out_of_stock_flags: list[str] = []
    misplaced_sku_list: list[MisplacedSku] = []
    price_tag_mismatch_count: int = 0


class Page(BaseModel):
    items: list[CompliancePayload]
    next_cursor: str | None = None


def _encode_cursor(ts: datetime, fixture_id: str) -> str:
    raw = json.dumps({"ts": ts.isoformat(), "fx": fixture_id}).encode()
    return base64.urlsafe_b64encode(raw).decode()


def _decode_cursor(cursor: str) -> tuple[datetime, str]:
    try:
        raw = json.loads(base64.urlsafe_b64decode(cursor.encode()))
        return datetime.fromisoformat(raw["ts"]), raw["fx"]
    except Exception as exc:  # malformed cursor must never 500
        raise HTTPException(status_code=400, detail="invalid cursor") from exc


@app.get("/v1/stores/{store_id}/compliance", response_model=Page)
async def list_compliance(
    store_id: str,
    since: date,
    until: date,
    cursor: str | None = None,
    limit: Annotated[int, Query(ge=1, le=200)] = 50,
    repo: "PayloadRepo" = Depends(get_repo),
    _scope: None = Depends(require_scope("compliance:read")),
) -> Page:
    """List compliance payloads for a store over a date range, newest first.

    Keyset pagination on (capture_timestamp, fixture_id) so pages stay stable
    while new scores land — an offset would skip or repeat rows under writes.
    """
    if until < since:
        raise HTTPException(status_code=422, detail="until precedes since")
    after = _decode_cursor(cursor) if cursor else None
    rows = await repo.fetch_page(store_id, since, until, after=after, limit=limit + 1)

    has_more = len(rows) > limit
    rows = rows[:limit]
    items = [CompliancePayload.model_validate(r) for r in rows]
    next_cursor = (
        _encode_cursor(items[-1].capture_timestamp, items[-1].fixture_id)
        if has_more and items else None
    )
    return Page(items=items, next_cursor=next_cursor)

The repository layer behind Depends(get_repo) is where the read cost is paid, and the one rule that keeps this endpoint fast is that a page of N payloads must cost one query, not N + 1. The rollup fields — a store’s aggregate compliance_percentage over a date range, its top misplaced SKUs — are served from a separate pre-aggregated table refreshed by the write path, never computed by looping over individual payloads inside the request. The keyset (cursor) pagination is not a stylistic choice: because new scores are constantly being written, an OFFSET-based page would skip or duplicate rows as the underlying set shifts, so the cursor encodes the last (capture_timestamp, fixture_id) seen and the query resumes strictly after it.

Production Configuration & Tuning Jump to heading#

Everything that governs the contract’s stability and the API’s blast radius lives in configuration, not code, so that a rate limit or a scope can change without a redeploy. The versioning strategy is the spine: the major version lives in the URL path (/v1/), and the exact minor version travels in the payload_version field and an X-Payload-Version response header. Additive minor bumps ship on /v1/; only a breaking change mints /v2/, and both major versions serve in parallel for a defined deprecation window.

compliance_api:
  versioning:
    current_major: "v1"
    current_minor: "1.4"
    supported_majors: ["v1"]        # add "v2" only for a breaking change
    deprecation_window_days: 180    # a retired major stays live this long
  pagination:
    default_page_size: 50
    max_page_size: 200              # hard ceiling; requests above are clamped
    cursor_ttl_hours: 24            # decoded cursors older than this are rejected
  rate_limits:
    "compliance:read":   { rpm: 600,  burst: 120 }
    "compliance:export": { rpm: 60,   burst: 20 }   # bulk pulls, tighter cap
    webhook_delivery:    { rps: 50 }                # outbound push ceiling
  auth:
    scopes: ["compliance:read", "compliance:export", "compliance:admin"]
    token_ttl_minutes: 30
    require_scope_per_route: true

The numbers are chosen against real consumer behavior. A default_page_size of 50 with a max_page_size of 200 keeps a single response under a few hundred kilobytes so p95 latency stays bounded; requests asking for more are clamped rather than rejected, because a BI tool paginating a year of history should degrade gracefully, not error. The read scope gets a generous 600 rpm because dashboards poll, while the export scope that pulls bulk history is capped at 60 rpm and separated precisely so a nightly ERP export can never starve interactive dashboard traffic. Auth scopes are enforced per route: a workforce-management token holds only compliance:read, an analytics ETL job holds compliance:export, and nothing but an internal admin process holds compliance:admin. The cursor_ttl_hours of 24 bounds how stale a paginating client’s view may be before it must restart — a consumer that holds a cursor for days is reading a snapshot that no longer reflects the store’s resets. Webhook delivery is rate-limited on the outbound side too, at 50 rps, so a chain-wide reset that scores thousands of fixtures at once cannot turn into a delivery storm; the throttling and coalescing of those bursts is the domain of the sibling Real-Time Compliance Alerting & Webhooks cluster, which this component feeds.

Failure Modes & Debugging Workflow Jump to heading#

When a consumer reports bad or missing data, the fault is almost always one of five recurring problems. Work them in this order:

1. Contract drift between the stored payload and the response model. Symptom: intermittent 500s on specific fixtures, or a Pydantic ValidationError in the service logs when serializing certain records. Root cause: the scoring stage started emitting a field shape the API’s response model does not accept — a misplaced_sku_list element missing detected_sku, or a compliance_percentage above 100. Fix: treat the response model as a gate on the write path too, validating payloads against the schema before they are stored, so drift fails at ingestion where it is cheap to fix rather than mid-request where it corrupts a consumer’s page.
2. An unversioned breaking change. Symptom: an ERP that worked yesterday now silently drops rows or misreads a field, with no error on either side. Root cause: a field was renamed or its type narrowed on /v1/ without bumping the major version, so consumers parsing the old shape quietly misparse. Fix: revert the change on /v1/, reintroduce it under /v2/, and enforce a schema-compatibility check in CI that fails any pull request narrowing a v1 field. This is the failure the whole versioning strategy exists to prevent, and the reset-aware rules for it live in Versioning Compliance Payloads Across Store Resets.
3. N+1 rollup queries. Symptom: list latency degrades linearly with page size, and the database shows one aggregate query per returned payload. Root cause: a rollup field is being computed inside the response loop instead of read from the pre-aggregated table. Fix: move the aggregate to the write path, serve it from the rollup table, and add a test that asserts a page of 50 issues exactly one query. This is the single most common cause of a read path that “works in staging” and collapses under a real store’s history.
4. Stale registry_revision in served payloads. Symptom: a fixture’s compliance reads as a cliff-edge collapse across many stores at once, and misplaced_sku_list is full of SKUs that legitimately belong there. Root cause: the payload was scored against a registry_revision that predates a planogram reset, so the API is faithfully serving a score that was computed against a stale planogram. This is not an API bug — the API’s job here is to make it visible: surface registry_revision in every payload and let consumers filter on it, and resolve the underlying drift upstream in Planogram Sync & SKU Mapping Strategies.
5. Pagination cursor gaps. Symptom: a consumer paginating a range reports missing or duplicated fixtures near page boundaries. Root cause: the keyset cursor is built on a non-unique sort key — capture_timestamp alone — so two fixtures captured in the same second straddle a page edge. Fix: make the cursor a compound key of (capture_timestamp, fixture_id) and resume strictly after both, which is exactly why the cursor above encodes both fields. A cursor decoded past its cursor_ttl_hours should return 400, never a silent partial result.

Keep a labeled repository of consumer-reported incidents bucketed by these five causes; the bucket that dominates tells you whether to harden the write-path schema gate, tighten CI compatibility checks, or fix the rollup path, and it is the only durable way to stop the same contract break from recurring after every schema change.

Scaling & Performance Benchmarks Jump to heading#

This component is read-dominated — writes arrive at the pace of captures, but reads scale with the number of dashboards, ERP polls, and BI refreshes across the enterprise — so the architecture optimizes the read path hard. Serve all list and rollup traffic from read replicas, keeping the primary for the scoring stage’s writes; the payload’s write-once immutability means replica lag is harmless for reads, because a payload never changes after it lands and a few seconds of lag only delays a fixture’s first appearance, never mutates one already served. Design against a soft SLA of p95 < 250 ms for a filtered, paginated list request and p95 < 120 ms for a cached store-level rollup, and alarm on the p95, not the mean, because the tail is where an un-indexed date filter or an N+1 rollup hides.

Two caches carry most of the load. Rollup aggregates are materialized on the write path and cached with a short TTL keyed by (store_id, date_range, payload_version), so a store’s headline compliance number is a single memory hit rather than an aggregation scan; the payload_version in the cache key is essential so a version rollout never serves a 1.4-shaped aggregate to a 1.5 consumer. Individual payloads are immutable, so they can be cached indefinitely by their (store_id, fixture_id, capture_timestamp) identity with no invalidation logic at all — the cheapest cache there is. The list endpoint’s cost is bounded by keyset pagination: because each page resumes strictly after the last cursor on an index over (store_id, capture_timestamp, fixture_id), page cost is constant regardless of how deep into a year of history a consumer reads, unlike an OFFSET scan whose cost grows with depth. Under a chain-wide morning reset, read traffic spikes as every category manager’s dashboard refreshes at once; that spike is absorbed by adding stateless API replicas behind the load balancer and by the rollup cache, both of which scale horizontally because the payload store is the only stateful tier and it is read-only on this path. The drift analytics that scan long histories for trend — a heavier, scan-oriented read pattern — run against the same replicas but off the interactive path, and their windowing is the concern of the sibling Time-Series Compliance Drift Analysis cluster. The concrete route-level shape of the interactive endpoints — status codes, error envelopes, content negotiation — is specified in Designing a Compliance Score REST API.

Frequently Asked Questions Jump to heading#

Related Jump to heading#

• Designing a Compliance Score REST API — the route-level specification of endpoints, status codes, and error envelopes for the API sketched here
• Versioning Compliance Payloads Across Store Resets — the reset-aware rules for what counts as an additive versus a breaking payload change
• Real-Time Compliance Alerting & Webhooks — the sibling cluster that consumes this component’s webhook surface and coalesces reset-time delivery storms
• Time-Series Compliance Drift Analysis — the scan-oriented read pattern that trends the same payloads over long histories
• Planogram Sync & SKU Mapping Strategies — the upstream pillar that produces the typed compliance payload this component stores and serves