Dataset: febrl4. Generated by make eval-benchmark. This file is committed and regenerated on release. FEBRL4 (dataset4a/dataset4b), fetched from the recordlinkage package at commit b93d97641952f8c85106be5794ca93b1f1298fbc and verified against pinned SHA-256 digests. Ground truth is not authored here: it is derived from the upstream record ids, where rec-N-org and rec-N-dup-0 are the same person. The records are generated by FEBRL's dsgen from real Australian name and address frequency tables, not collected from real people.
A false merge joins two different people and can corrupt a record irreversibly. A missed match leaves a duplicate, which is recoverable. The two errors are not equal, so the gated metric is the false-merge rate among auto-merged pairs, and it is reported with a Wilson confidence interval because the denominator is small.
| Metric | Value | 95% CI |
|---|---|---|
| Records | 10000 | |
| True duplicate pairs (ground truth) | 5000 | |
| Scored pairs kept (above the reporting floor) | 41136 | |
| Auto-merged pairs | 3714 | |
| Pairs sent to review | 712 | |
| False-merge rate (gated) | 0.0% (0/3714) | [0.0%, 0.1%] |
| Missed-match rate | 12.1% (603/5000) | [11.2%, 13.0%] |
| Precision, auto | 100.0% | |
| Recall, auto | 74.3% | |
| F1, auto | 85.2% | |
| Precision, auto+review coverage | 99.3% | |
| Recall, auto+review coverage | 87.9% | |
| F1, auto+review coverage | 93.3% | |
| True pairs never scored | 61 |
A true pair goes unscored two ways, and this row counts both: blocking never generated it, or blocking generated it and the matcher scored it below the 0.001 floor that keeps near-zero pairs out of the result. The row was labelled a blocking count until v0.8, which read as though only the blocking rules could reach these pairs. They are separate causes with separate fixes, and on the external benchmark the second was by far the larger of the two.
False-merge gate at threshold 1.0%: PASS (observed 0.0%).
Recall at the auto level is intentionally below 100%: pairs the matcher is unsure about are not auto-merged, they are sent to review. The auto+review coverage recall is the share of true duplicates the system surfaces to a human one way or another.
F1 is reported for comparability with published record-linkage results and is not a gate. It weighs a false merge and a missed match equally, which this pipeline does not.
Not applicable to this run. The LLM field judge never executed: the corpus is structured CSV, so no extraction seam was invoked and there are no confidence verdicts for human labels to agree with. This is not a waived gate. Reporting a kappa failure for a component that did not run would be a false claim about coverage, the same way a fixed provenance sentence was; the gate itself still fails closed wherever the judge does run.
Confirmation that the scored records are the benchmark's and not a cached or fixture corpus. Digests are of the raw upstream files as downloaded; the pipeline read the converted form of exactly these bytes.
| Source file | SHA-256 |
|---|---|
dataset4a.csv |
07c7cb3f0a8d88180e80317f2a60499dee4e8324a44c38059f4e7fed0a8b4488 |
dataset4b.csv |
2eed76c99fa2237be3ec013a123427926d4158abcb3a8f65874d6c7f1358cf2c |
Upstream commit b93d97641952f8c85106be5794ca93b1f1298fbc. Records ingested: 10000. Ground-truth pairs derived from upstream record ids: 5000.
| Canonical field | Raw non-empty | Normalized non-empty | Dropped |
|---|---|---|---|
| first_name | 9654 | 9654 | 0 |
| last_name | 9850 | 9850 | 0 |
| dob | 9707 | 9643 | 64 |
| address | 10000 | 10000 | 0 |
FEBRL4 carries no email and no phone column, so those two canonical fields are absent from the corpus rather than empty in it.
Ground-truth duplicates the matcher auto-merged, named by their upstream ids so they can be looked up in the source files:
existing:rec-1262-org=incoming:rec-1262-dup-0existing:rec-1357-org=incoming:rec-1357-dup-0existing:rec-1730-org=incoming:rec-1730-dup-0