Calibrating IoU Thresholds for Dense Endcap Displays

This walkthrough sits under Position Validation Algorithms for Planograms and solves one precise task: choosing the iou_floor that grades a dense endcap correctly. The parent stage projects detections to a metric grid, assigns each to a slot_id, and grades every matched pair in_position, shifted, or misplaced — with a matched SKU whose overlap falls below iou_floor flipped to misplaced. That floor is easy to set on a clean modular bay where facings sit apart and each detection lands squarely inside one slot rectangle. It is a trap on a packed endcap. When products are stacked tight and their boxes physically abut, a correct facing can share overlap with two adjacent slot rectangles, so its slot_iou against its own slot is genuinely lower than the same SKU would score in open shelving. A floor inherited from a flat bay then mis-grades the neighbor as misplaced, and the endcap reads as a merchandising failure that never happened. This page builds a data-driven calibration — one distribution per fixture class, one swept floor, one auditor-agreement metric — so the floor you ship is the one that actually matches a human verdict.

Prerequisites & Context Jump to heading#

The floor is only meaningful once the geometry upstream of it is trustworthy. Confirm all of the following before you sweep a single value.

• Runtime: Python 3.11+ with numpy on the scoring host; no GPU is needed — calibration runs on already-computed overlaps, not images.
• Slot-mapped detections carrying slot_iou: each detection must arrive already assigned to a slot_id with its Intersection-over-Union against the slot rectangle and its centroid_offset_mm, exactly the typed record the parent stage emits. If slot_iou is still computed in pixel space it is not comparable across fixtures — resolve that with the homography pass in the parent page first.
• A labeled endcap audit set: a few hundred slots from real endcaps, each tagged by a human as correct facing present or not, and each stamped with its fixture_id and capture_timestamp. Without ground truth there is nothing to agree with, and the loop that formalizes this reconciliation lives in Threshold Tuning for Compliance Accuracy.
• A fixture-class label per slot: every audited slot must name its fixture class — curved_endcap, flat_endcap, wing_rack, dump_bin — because the whole argument of this page is that these classes have different slot_iou distributions and cannot share one floor.
• Metric offsets alongside the IoU: the same centroid_offset_mm the parent stage produces, so the Step 4 tie-break has a distance signal to lean on when overlap alone is ambiguous.

A note on scope: this page tunes the identity gate — is the correct product actually in this slot — not the tolerance bands that separate in_position from shifted. Those bands are the subject of the sibling walkthrough, Validating Shelf Position Tolerances in Retail; read the two together, because a floor set here and a band set there must not contradict each other.

Step 1 — Build a slot_iou Distribution per Fixture Class Jump to heading#

Never eyeball a floor from a single number. Build the empirical distribution of slot_iou for each fixture class from the labeled set, split by the auditor’s verdict, and look at where the two populations separate. On a dense endcap the two humps overlap far more than on a flat bay: correctly placed products score lower because their boxes abut a neighbor’s slot, and foreign products score a little higher because tight packing pushes everything into everyone’s rectangle. The distance between the humps — not any textbook default — is what a defensible floor sits in.

from __future__ import annotations

from dataclasses import dataclass

import numpy as np


@dataclass(frozen=True)
class AuditSample:
    """One slot from a labeled endcap audit set."""

    fixture_class: str          # e.g. "curved_endcap", "flat_endcap"
    slot_iou: float             # detection-vs-slot overlap from the parent stage
    centroid_offset_mm: float   # metric distance to the slot anchor
    auditor_ok: bool            # True if a human graded the correct facing present


def iou_distribution(
    samples: list[AuditSample], fixture_class: str, bins: int = 20
) -> tuple[np.ndarray, np.ndarray]:
    """Histogram of slot_iou for one fixture class over the [0, 1] range."""
    ious = np.array(
        [s.slot_iou for s in samples if s.fixture_class == fixture_class]
    )
    if ious.size == 0:
        raise ValueError(f"no audit samples for fixture_class={fixture_class!r}")
    if ious.min() < 0.0 or ious.max() > 1.0:
        raise ValueError("slot_iou values must lie in [0, 1]")
    counts, edges = np.histogram(ious, bins=bins, range=(0.0, 1.0))
    return counts, edges

If the two verdicts do not visibly separate anywhere on the axis, no floor will grade this fixture class well and the problem is upstream — the assignment is confusing neighbors before the floor ever sees them. Fix that in the parent stage rather than hunting for a magic threshold here.

Step 2 — Sweep iou_floor and Maximize Auditor Agreement Jump to heading#

With the distributions in hand, do not pick the visual crossover point by hand — sweep the floor across a range and score each candidate against the auditor labels with a single agreement metric. F1 is the right choice because the two errors are asymmetric and both matter: a floor set too high converts correct-but-tight facings into false misplaced records (hurting recall), while a floor set too low lets foreign products pass the gate (hurting precision). The floor that maximizes F1 is the one whose grades most often match a human, which is the only definition of “correct” that survives a category-manager review.

@dataclass(frozen=True)
class SweepPoint:
    iou_floor: float
    f1: float
    precision: float
    recall: float


def sweep_iou_floor(
    samples: list[AuditSample],
    fixture_class: str,
    lo: float = 0.10,
    hi: float = 0.70,
    step: float = 0.02,
) -> list[SweepPoint]:
    """Sweep iou_floor and score agreement (F1) against auditor labels."""
    subset = [s for s in samples if s.fixture_class == fixture_class]
    if not subset:
        raise ValueError(f"no samples for fixture_class={fixture_class!r}")
    if not 0.0 <= lo < hi <= 1.0:
        raise ValueError("require 0 <= lo < hi <= 1")

    points: list[SweepPoint] = []
    for floor in np.arange(lo, hi + 1e-9, step):
        tp = fp = fn = 0
        for s in subset:
            predicted_ok = s.slot_iou >= float(floor)
            if predicted_ok and s.auditor_ok:
                tp += 1
            elif predicted_ok and not s.auditor_ok:
                fp += 1
            elif not predicted_ok and s.auditor_ok:
                fn += 1
        precision = tp / (tp + fp) if tp + fp else 0.0
        recall = tp / (tp + fn) if tp + fn else 0.0
        f1 = (
            2 * precision * recall / (precision + recall)
            if precision + recall
            else 0.0
        )
        points.append(SweepPoint(round(float(floor), 3), f1, precision, recall))
    return points


def best_floor(points: list[SweepPoint]) -> SweepPoint:
    """Floor with maximum F1; ties broken toward the lower (more forgiving) floor."""
    if not points:
        raise ValueError("empty sweep; nothing to choose")
    return max(points, key=lambda p: (p.f1, -p.iou_floor))

On a typical curved_endcap this sweep lands the optimum around iou_floor 0.34 — well below the 0.30-to-0.50 range a flat bay tolerates, and low enough that a correct facing sharing overlap with its neighbor still clears the gate. The tie-break toward the lower floor in best_floor is deliberate: when two candidates score identical F1, the more forgiving one produces fewer false misplaced alerts, which is the cheaper error to make on a promotional fixture that a merchandiser is watching.

Step 3 — Add a Fixture-Class Override, Not a Global Value Jump to heading#

The mistake that follows a good sweep is shipping its single winning number as the platform-wide iou_floor. A floor tuned on curved_endcap will over-reject on a flat_endcap and under-reject on a dump_bin. Calibrate every class independently and emit an override map keyed by fixture class, with a conservative default for classes too sparse to tune. This is the same override pattern the parent stage uses for curved_endcap in its assignment config — you are extending it, not inventing it.

def calibrate_per_class(
    samples: list[AuditSample],
    default_floor: float = 0.30,
    min_samples: int = 40,
) -> dict[str, float]:
    """Per-fixture-class iou_floor override with a default fallback."""
    overrides: dict[str, float] = {"_default": default_floor}
    classes = {s.fixture_class for s in samples}
    for fc in sorted(classes):
        subset = [s for s in samples if s.fixture_class == fc]
        if len(subset) < min_samples:
            # too little ground truth to trust a swept value; use the default
            overrides[fc] = default_floor
            continue
        chosen = best_floor(sweep_iou_floor(samples, fc))
        overrides[fc] = chosen.iou_floor if chosen.f1 > 0.0 else default_floor
    return overrides


if __name__ == "__main__":
    rng = np.random.default_rng(7)
    audit: list[AuditSample] = []
    # correct facings on a curved endcap: overlap depressed by tight packing
    for _ in range(160):
        audit.append(AuditSample("curved_endcap", float(np.clip(rng.normal(0.55, 0.11), 0, 1)),
                                 float(abs(rng.normal(6, 4))), True))
    # foreign / misplaced products: lower overlap
    for _ in range(90):
        audit.append(AuditSample("curved_endcap", float(np.clip(rng.normal(0.20, 0.09), 0, 1)),
                                 float(abs(rng.normal(30, 10))), False))
    overrides = calibrate_per_class(audit)
    print(f"curved_endcap floor = {overrides['curved_endcap']}  (default {overrides['_default']})")

Version this map alongside the registry_revision it was tuned against so a floor change is auditable and reversible. When a new endcap style appears in a chain reset it inherits _default until it has enough labeled slots to earn its own row — never a value borrowed from a visually similar class. Endcaps that run promotional campaigns route their geometry through Promotional Display Alignment Checks, whose per-campaign schema can carry its own floor on top of this map.

Step 4 — Break Ties With Centroid Distance, Not IoU Alone Jump to heading#

Even a well-calibrated floor leaves a thin ambiguous band — the shaded region in the diagram — where a correct facing and a foreign one score nearly the same slot_iou. On packed rows of identical or look-alike SKUs this is where mistakes cluster, because two tight neighbors can each overlap the same slot rectangle by almost the same amount. The fix is to stop asking overlap to decide alone: inside a small band around the floor, let centroid_offset_mm cast the deciding vote, since the correctly placed facing is the one whose centre actually sits nearest the slot anchor.

def grade_facing(
    detected_sku: str,
    expected_sku: str,
    slot_iou: float,
    centroid_offset_mm: float,
    iou_floor: float,
    tie_band: float = 0.05,
    shift_mm: float = 12.0,
) -> str:
    """Grade one facing, using centroid distance to break IoU ties on packed rows."""
    if detected_sku != expected_sku:
        return "misplaced"
    if slot_iou < iou_floor - tie_band:
        return "misplaced"
    if slot_iou < iou_floor + tie_band:
        # ambiguous overlap: the nearer centroid decides identity
        return "in_position" if centroid_offset_mm <= shift_mm else "misplaced"
    return "in_position" if centroid_offset_mm <= shift_mm else "shifted"

Keeping the distance signal separate from the overlap signal is what stops a tight neighbor from being graded misplaced on overlap alone. The tie_band of 0.05 is intentionally narrow: widen it and centroid distance starts overriding overlap for facings that were never ambiguous, which reintroduces the pixel-fragile behavior the metric frame was built to remove.

Verification & Testing Jump to heading#

Confirm the calibration deterministically rather than trusting the dashboard to look reasonable.

1. The chosen floor maximizes F1. Assert that best_floor(sweep_iou_floor(...)) returns the sweep point with the highest F1, and that no other point in the returned list has a strictly greater f1.
2. Adjacent correct facings are not misgraded. Build a synthetic curved_endcap where correct facings cluster at slot_iou 0.40 and foreign ones at 0.18; assert the calibrated floor lands between them and that grading the 0.40 facings yields zero misplaced records.
3. The tie-break rescues a near neighbor. Call grade_facing with the correct SKU, a slot_iou one hundredth below iou_floor, and a centroid_offset_mm of 4.0; assert it returns in_position, then raise the offset past shift_mm and assert it flips to misplaced.
4. Per-class overrides differ from the global default. Assert calibrate_per_class returns a curved_endcap floor distinct from _default, and that a class below min_samples falls back to the default exactly.
5. Degenerate input fails loud. Assert iou_distribution and sweep_iou_floor raise ValueError on an unknown fixture class and on slot_iou values outside [0, 1], so a malformed audit set never silently produces a floor of 0.0.

A healthy result shows the in_position share on audited endcaps tracking the auditor pass rate within a few points, with misplaced records dominated by genuine wrong-SKU cases rather than tight correct neighbors clipped by an inherited floor.

Troubleshooting Jump to heading#

Related Jump to heading#

• Position Validation Algorithms for Planograms — the parent stage that computes slot_iou and applies the iou_floor this page calibrates
• Validating Shelf Position Tolerances in Retail — the sibling walkthrough that tunes the distance bands separating in_position from shifted
• Threshold Tuning for Compliance Accuracy — the reconciliation loop that formalizes agreeing floors against ground-truth audits
• Promotional Display Alignment Checks — endcap and secondary-display geometry that layers a per-campaign schema over the fixture-class floor