Skip to content

Latest commit

 

History

History
27 lines (17 loc) · 1.62 KB

File metadata and controls

27 lines (17 loc) · 1.62 KB

Eval report

Dataset: housing-demo. Generated by receipts eval. This file is committed and regenerated on release. The fixtures are seeded synthetic service data with planted ground-truth figures; there is no real personal data.

Why this metric

A number in a funder report that is wrong or invented is the expensive, sometimes irreversible error. So the gated metric is the grounding rate: the share of numbers in the narrative that bind to a receipt. It is fail-closed at 100%; a single unbound number blocks export.

What was scored

The publishable figure set: the narrative is drafted and grounded after small-cell suppression, so this scores the artifact receipts run exports rather than a pre-suppression draft the pipeline would never produce. A suppressed cell renders as [SUPPRESSED] and carries no number, so it contributes no span to the denominator. The denominator is therefore the count of numbers that survive suppression, which is smaller than the spec's metric count whenever a report has a small cell.

Results

Metric Value 95% CI
Numbers in narrative 1
Bound to a receipt 1
Grounding rate (gated) 100.0% (1/1) [20.7%, 100.0%]
Unverifiable numbers 0
Hallucinated-number rate 0.0% (0/1) [0.0%, 79.3%]

Gate

Grounding gate (100% required): PASS (observed 100.0%).

This committed run scores the drafted narrative, every number of which comes from a receipt, so it passes. That the gate catches an injected unverifiable number is shown by the merge-blocking test tests/test_grounding_gate.py, not by failing this report.