Dataset: intake-demo. Generated by reconcile eval. This file is committed and regenerated on release. Numbers come from running the matcher on seeded synthetic fixtures with planted ground truth; there is no real personal data in the fixtures.
A false merge joins two different people and can corrupt a record irreversibly. A missed match leaves a duplicate, which is recoverable. The two errors are not equal, so the gated metric is the false-merge rate among auto-merged pairs, and it is reported with a Wilson confidence interval because the denominator is small.
| Metric | Value | 95% CI |
|---|---|---|
| Records | 27 | |
| True duplicate pairs (ground truth) | 7 | |
| Scored pairs kept (above the reporting floor) | 12 | |
| Auto-merged pairs | 6 | |
| Pairs sent to review | 2 | |
| False-merge rate (gated) | 0.0% (0/6) | [0.0%, 39.0%] |
| Missed-match rate | 0.0% (0/7) | [0.0%, 35.4%] |
| Precision, auto | 100.0% | |
| Recall, auto | 85.7% | |
| F1, auto | 92.3% | |
| Precision, auto+review coverage | 87.5% | |
| Recall, auto+review coverage | 100.0% | |
| F1, auto+review coverage | 93.3% | |
| True pairs never scored | 0 |
A true pair goes unscored two ways, and this row counts both: blocking never generated it, or blocking generated it and the matcher scored it below the 0.001 floor that keeps near-zero pairs out of the result. The row was labelled a blocking count until v0.8, which read as though only the blocking rules could reach these pairs. They are separate causes with separate fixes, and on the external benchmark the second was by far the larger of the two.
False-merge gate at threshold 0.0%: PASS (observed 0.0%).
Recall at the auto level is intentionally below 100%: pairs the matcher is unsure about are not auto-merged, they are sent to review. The auto+review coverage recall is the share of true duplicates the system surfaces to a human one way or another.
F1 is reported for comparability with published record-linkage results and is not a gate. It weighs a false merge and a missed match equally, which this pipeline does not.
Cohen's kappa measures agreement between the extractor's confidence verdicts and human field labels on a committed calibration fixture. Kappa below 0.60 means confidence scores are not tracking accuracy well enough to trust, so the gate fails.
Cohen's kappa: 0.80 over 20 labels.
Kappa gate at 0.60: PASS (observed 0.80).