forked from ChelseaKR/outcome-receipts
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathsuppression.py
More file actions
659 lines (558 loc) · 31.1 KB
/
Copy pathsuppression.py
File metadata and controls
659 lines (558 loc) · 31.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
"""Small-cell suppression: privacy-protecting redaction of aggregate counts.
Suppression logic is modeled on the U.S. CMS Cell Size Suppression Policy:
- Aggregate counts in the range [1, 10] (below the threshold of 11) are suppressed.
- True zeros (count = 0) are preserved unencrypted, as they contain no privacy risk.
- Complementary suppression is applied: when a cell is suppressed, other cells in
the same crosstab are suppressed as needed so the suppressed value cannot be
recovered by subtraction. For single-dimension tables with no complementary cells,
the suppressed value alone is redacted.
A suppressed ``Figure`` is redacted at every layer that could leak the raw count:
its own ``value``/``display``, and its ``Receipt``'s ``value``, ``row_count``, and
``slice_hash``. A caller that reads ``figure.receipt.row_count`` (as the report,
manifest, and trace renderers do) must see the same redaction a caller reading
``figure.value`` sees; a suppressed ``Figure`` sharing an unredacted ``Receipt`` is
not suppression, it is a suppressed label glued to an unsuppressed number.
Disclosure scope: the whole report, per unit. An earlier revision scoped the
complementary check to sibling groups (a comparison metric's period and delta
figures together, every headline metric in a separate report-level group), on
the theory that only figures presented together would be combined by a reader.
That partition severed real accounting identities: a whole-period headline is
the sum of its own period figures (``exits_permanent = __q1 + __q2``), so
``headline - q2`` printed a suppressed ``q1`` into the same report, undetected,
because the two figures sat in different groups. The report spec is a flat
metric list over one data table; per-period category sums, category totals, and
headline/period identities all cross any finer grouping the metric ids could
induce, so any partition can sever an identity a reader actually knows. The
rule now: every count figure in a report is checked against every other count
figure, and every percent figure against every other percent figure (counts and
percents are never additive with each other). The cost is that a coincidental
arithmetic match between unrelated figures can suppress more than strictly
necessary; over-suppression is the protective direction, and a leak is a
defect, so the trade is accepted.
Two relationships need rules of their own, because they are not plain +/- sums:
- A comparison delta is defined as current minus prior. When either period
figure is suppressed, the delta is suppressed with it: a visible delta beside
the other period reconstructs the hidden period directly, and a visible delta
beside a visible whole-period headline pins the hidden period at
``(headline - delta) / 2`` -- a recovery with a coefficient the signed-sum
search cannot represent, so it is closed by rule rather than by search.
- A percent is a ratio of counts, and a percent with a visible denominator
uniquely determines a suppressed numerator via rounding (71% of 14 exits can
only be 10). ``MetricSpec`` cannot express which count metrics feed a percent
(``value_sql`` is opaque SQL), so the conservative rule applies: when any
count figure in the report is suppressed, every percent figure is suppressed
with it. This is deliberately blunt -- a percent whose numerator and
denominator are both visible is suppressed too -- because the dependency
cannot be traced from the data model, and guessing it from SQL text is the
name-matching heuristic this module already rejected once.
The disclosure boundary is not limited to ``Figure``s. A ``ComparisonRow``'s
``direction`` (and its derived ``arrow``) is a word computed from the sign of
the raw, unredacted delta, not a ``Figure`` with its own value or receipt --
so the exhaustive same-unit search above, which ranges only over ``Figure``
values, cannot see it, redact it, or use it as a candidate. Left alone, it
asserts a real fact about a cell the report declined to publish: "no change"
certifies prior == current *exactly*. ``redact_comparison`` and
``redact_reconciliation`` close this by direct rule rather than by search --
whenever a row's ``prior``, ``current``, or ``delta`` ends up redacted, that
row's ``direction`` is redacted with it, to the same sentinel a redacted
``Figure`` displays. See ``_redact_row_direction``. Any future presentation
field derived from a figure's raw value, rather than carried as a ``Figure``
itself, needs the same treatment: it is inside this boundary by default, not
outside it by omission.
The guarantee, stated precisely: after suppression reaches its fixed point, no
suppressed figure's exact value is certified to a reader by the figures still
visible -- not by any single +/- combination of same-unit visible figures, not
by a delta definition, and not by a percent with a visible input. Two things
are deliberately outside that guarantee. A suppressed value may still be
*consistent* with the visible figures (a headline of 13 with both period
figures hidden tells the reader the two hidden values sum to 13; an interval
is not a disclosure under the cell-suppression model). And a hidden value may
coincidentally equal some arithmetic on visible figures from unrelated metrics
(nothing certifies the coincidence to the reader, who cannot distinguish it
from the other combinations that do not match). What must never happen is the
demonstrated failures: an accounting identity the reader actually holds --
total minus categories, headline minus a period, one period plus a visible
delta, a percent times its visible denominator -- landing exactly on a
suppressed cell.
Policy basis:
CMS primary data documentation states that values 1--10, derivable cells, and
revealing percentages/formulas may not be displayed:
https://data.cms.gov/sites/default/files/2023-11/51397ef0-8f37-40f6-985f-4a46c61882cb/Data_Dictionary-MSSP-Performance_Year_Financial_and_Quality_Results__2013-2020.pdf
HUD's HMIS publication guidance requires anonymous aggregate public data and
avoidance of small-sample inference, but does not prescribe a numeric floor:
https://files.hudexchange.info/resources/documents/HMISImplementationGuide.pdf
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass, field, replace
from typing import TYPE_CHECKING
from outcome_receipts.models import REDACTED_DISPLAY, Figure
if TYPE_CHECKING:
from collections.abc import Sequence
from outcome_receipts.comparison import ComparisonResult, ComparisonRow, ReconciliationResult
# The CMS Cell Size Suppression Policy threshold: counts below this value
# (i.e., 1-10) are suppressed.
SUPPRESSION_THRESHOLD = 11
# The redacted placeholder shown in place of a suppressed figure's value.
_REDACTED_DISPLAY = REDACTED_DISPLAY
# Tolerance for treating a candidate combination's sum as equal to the target.
# Figures are counts (or percents computed to fixed decimals), so exact-match
# with a float epsilon is the right test; nothing here is approximate.
_EPSILON = 1e-9
@dataclass(frozen=True)
class SuppressionResult:
"""The outcome of suppression over a figure set.
``suppressed`` are the metric_ids of figures whose counts fell below the
threshold and were redacted. ``complementary_suppressed`` are the metric_ids
of figures redacted via complementary suppression to prevent recovery of a
suppressed value. ``unsuppressed`` are the metric_ids of figures that passed
unredacted (either above threshold or true zeros). ``aggregate_only`` is True
when no row-level data was emitted in the export (the privacy assertion).
``threshold`` and ``values`` are kept so ``ok`` can check the actual privacy
invariant against the figures suppression saw, rather than against a count of
ids that says nothing about whether any of them were below threshold.
"""
suppressed: tuple[str, ...] = field(default_factory=tuple)
complementary_suppressed: tuple[str, ...] = field(default_factory=tuple)
unsuppressed: tuple[str, ...] = field(default_factory=tuple)
aggregate_only: bool = True
threshold: int = SUPPRESSION_THRESHOLD
# metric_id -> the figure's original (pre-redaction) value, for every figure
# suppress_figures was given. Not a public reporting field; it exists so `ok`
# can verify the invariant it claims to check.
values: tuple[tuple[str, float], ...] = field(default_factory=tuple)
@property
def ok(self) -> bool:
"""True only if no cell below threshold remains unsuppressed.
This checks the actual privacy invariant: every metric_id in
``unsuppressed`` must have an original value that is either a true zero
or at or above ``threshold``. A metric_id with no recorded value (e.g. a
``SuppressionResult`` built by hand, as some tests do) is assumed fine,
since there is nothing to check it against.
"""
by_id = dict(self.values)
for metric_id in self.unsuppressed:
value = by_id.get(metric_id)
if value is not None and 1 <= abs(value) < self.threshold:
return False
return True
def _is_below_threshold(value: float, threshold: int) -> bool:
"""True if ``value``'s magnitude is a small cell: in [1, threshold).
Magnitude, not the raw signed value, is what determines suppression. Every
ordinary count is non-negative, so this is a no-op for them. It matters for a
comparison's delta figure, whose value is a signed difference: a swing of
"-1" is exactly as disclosive as a swing of "1" (it still names a change of
one person), so a negative small delta must be suppressed too, not waved
through because ``value < 1`` reads False for a negative number.
"""
return 1 <= abs(value) < threshold
def _redact(figure: Figure) -> Figure:
"""A copy of ``figure`` with every raw-count-bearing field withheld.
Redacting only ``Figure.value``/``display`` and leaving ``figure.receipt``
attached unchanged is not suppression: ``report.py`` and ``trace.py`` read
``receipt.row_count`` directly, and both render it right next to the
"[SUPPRESSED]" label if the receipt is not also redacted. So every field on
the receipt that carries the raw count is replaced here.
They are replaced with ``None``, not with zero, and the receipt is stamped
``suppressed=True``. Zeroing them was the earlier behaviour and it made a
withheld cell byte-identical to a true zero in every field the manifest
schema constrains: ``value: 0.0``, ``row_count: 0``, and the all-zero
``EMPTY_SLICE_HASH`` are exactly what a figure of genuinely zero produces.
The prose said ``[SUPPRESSED]``; the numbers said nobody. Every machine
consumer -- the manifest, the trace view, the six evidence workflows -- read
the numbers. ``None`` serialises as JSON ``null``, so a consumer that sums or
plots the field fails loudly instead of silently counting a withheld group as
zero, and ``suppressed`` gives it a field to branch on.
The row count is withheld rather than kept, even though it is the one honest
fact a redacted receipt could still carry ("a query ran and matched rows"),
because for an ordinary count metric the row count *is* the suppressed value.
``suppressed: true`` already tells a reader, under the documented policy,
that a query ran; publishing the count would tell them what it returned.
``value_sql``, ``unit``, ``computed_at``, and ``definition`` are not data;
they describe the query and are kept for audit purposes.
"""
redacted_receipt = replace(
figure.receipt,
row_count=None,
slice_hash=None,
value=None,
column_names=None,
suppressed=True,
)
# ``Figure.value`` goes with them. It is what the chart renderer reads for
# geometry, and a redacted figure carrying 0.0 drew a bar of height zero on
# the axis baseline and a line straight through the floor -- a picture of
# "we housed nobody this quarter" beside a label saying the opposite.
return Figure(
metric_id=figure.metric_id,
value=None,
display=_REDACTED_DISPLAY,
receipt=redacted_receipt,
)
def _disclosing_combination(
target: float, candidates: list[tuple[str, float]]
) -> tuple[str, ...] | None:
"""Find a signed combination of ``candidates`` that reconstructs ``target``.
Checks combinations of every size up to the full candidate set, with every
assignment of + or - to each term, e.g. "total - other_category". The search
used to stop at four terms, which let a five-category breakdown evade it:
the only combination recovering the suppressed fifth category
(``total - a - b - c - d``) has five terms, and breakdowns of five or more
categories are ordinary in HMIS-style reporting. No within-group identity
may escape on size, so the search is exhaustive over the group.
Implemented as a depth-first search over the candidates sorted by descending
magnitude, with branch-and-bound pruning: a branch is abandoned as soon as
the remaining candidates' combined magnitude cannot close the gap between
the partial sum and the target. Report figure sets are small (tens of
figures at most), so the pruned search is cheap in practice.
Returns the metric_ids of a combination that reconstructs ``target``, or
``None`` if no such combination exists among ``candidates``.
"""
# Largest magnitudes first, so pruning bites as early as possible. The
# secondary sort on metric_id keeps the search order, and therefore the
# returned combination, deterministic.
ordered = sorted(candidates, key=lambda item: (-abs(item[1]), item[0]))
values = [value for _, value in ordered]
# remaining[i]: the combined magnitude of candidates i..end, the most any
# completion of a partial combination can still move the sum.
remaining = [0.0] * (len(ordered) + 1)
for index in range(len(ordered) - 1, -1, -1):
remaining[index] = remaining[index + 1] + abs(values[index])
chosen: list[int] = []
def search(index: int, partial: float) -> bool:
if chosen and abs(partial - target) < _EPSILON:
return True
if index == len(ordered) or abs(partial - target) > remaining[index] + _EPSILON:
return False
for sign in (1.0, -1.0):
chosen.append(index)
if search(index + 1, partial + sign * values[index]):
return True
chosen.pop()
return search(index + 1, partial)
if search(0, 0.0):
return tuple(ordered[index][0] for index in chosen)
return None
# The suffix `comparison.py` gives a period-over-period delta figure's metric_id
# (see `_delta_spec`). A delta is *defined* as current minus prior, so it is
# always "recoverable" from its own two period figures -- that is not a
# disclosure the arithmetic check discovered, it is the delta's definition. If
# it were treated as an ordinary target, any suppressed delta whose own periods
# happen to still be visible (typically because both periods are safely at or
# above the threshold on their own) would cascade into suppressing one of those
# safe period figures for no privacy benefit, since the reader could already
# compute the "protected" delta from the two period values regardless of what
# the delta figure itself displays. So a delta's own suppression stands on its
# own (it still gets redacted if its magnitude is small); it just does not pull
# other figures down with it. The reverse dependency is real, though: once
# either of a delta's period figures is suppressed, the delta must go with it
# (see `_complementary_suppress`), because a visible delta reconstructs the
# hidden period from the other period, or -- combined with a visible
# whole-period headline -- pins it at (headline - delta) / 2.
_DELTA_SUFFIX = "__delta"
def _sibling_delta_id(metric_id: str) -> str | None:
"""The delta metric_id belonging to a comparison period figure, or None.
``exits__q1`` -> ``exits__delta``. A figure that is not suffixed, or is
itself a delta, has no sibling delta.
"""
if "__" not in metric_id or metric_id.endswith(_DELTA_SUFFIX):
return None
return metric_id.rsplit("__", 1)[0] + _DELTA_SUFFIX
def _suppress_sibling_deltas(
values_by_id: dict[str, float],
hidden: Callable[[str], bool],
complementary: set[str],
) -> bool:
changed = False
for metric_id in sorted(values_by_id):
delta_id = _sibling_delta_id(metric_id)
if delta_id is None or not hidden(metric_id):
continue
if delta_id in values_by_id and not hidden(delta_id):
complementary.add(delta_id)
changed = True
return changed
def _suppress_dependent_rates(
values_by_id: dict[str, float],
units_by_id: dict[str, str],
hidden: Callable[[str], bool],
hidden_ids: set[str],
complementary: set[str],
) -> bool:
if not any(units_by_id[mid] == "count" for mid in hidden_ids):
return False
victims = {
metric_id
for metric_id in values_by_id
if units_by_id[metric_id] in {"percent", "rate"} and not hidden(metric_id)
}
complementary.update(victims)
return bool(victims)
def _recovery_victim(
metric_id: str,
values_by_id: dict[str, float],
units_by_id: dict[str, str],
hidden: Callable[[str], bool],
) -> str | None:
if metric_id.endswith(_DELTA_SUFFIX) or metric_id not in values_by_id:
return None
candidates = [
(other_id, value)
for other_id, value in values_by_id.items()
if other_id != metric_id
and not hidden(other_id)
and units_by_id[other_id] == units_by_id[metric_id]
]
combo = _disclosing_combination(values_by_id[metric_id], candidates)
if combo is None:
return None
return min(combo, key=lambda mid: (abs(values_by_id[mid]), mid))
def _complementary_suppress(figures: list[Figure], suppressed_ids: set[str]) -> set[str]:
"""Disclosure-based complementary suppression: real arithmetic, not names.
Runs three rules to a shared fixed point, since applying any one of them can
expose work for another:
1. A suppressed period figure pulls its own delta figure down with it. The
delta is current minus prior by definition; visible, it reconstructs the
hidden period from the other period figure or from a whole-period
headline, and the headline route has a coefficient (a half) the signed
sum search below cannot represent. See the ``_DELTA_SUFFIX`` note.
2. Once any count figure is suppressed (primary or complementary), every
percent figure is suppressed too. A percent with a visible denominator
uniquely determines a suppressed numerator via rounding, and the data
model cannot say which counts feed which percent, so the conservative
rule from the module docstring applies.
3. For every primary-suppressed figure, the exact-recovery search: can the
suppressed value be rebuilt by adding or subtracting still-visible
figures of the same unit anywhere in the report (a total minus its other
named parts, a headline minus the other period, a sibling category minus
another)? If a disclosing combination exists, the smallest-valued figure
in it is suppressed -- the standard "next-smallest cell" rule, the least
additional information that breaks that recovery -- and the loop repeats,
since one redaction can leave another combination that discloses the same
or a different suppressed cell.
The disclosure scope is the whole report, split only by unit (counts check
against counts, percents against percents); see the module docstring for why
any finer grouping severed real accounting identities. A comparison delta
figure is never a *target* of rule 3 (its recoverability from its own
periods is definitional, not a discovered disclosure), but it is a candidate
while visible.
"""
complementary: set[str] = set()
# Every figure has a value here: ``suppress_figures`` refuses an input set
# containing a redacted one, so the filter narrows the type without
# discarding anything the search would otherwise have considered.
values_by_id = {
figure.metric_id: figure.value for figure in figures if figure.value is not None
}
units_by_id = {figure.metric_id: figure.receipt.unit for figure in figures}
def hidden(metric_id: str) -> bool:
return metric_id in suppressed_ids or metric_id in complementary
changed = True
while changed:
changed = False
changed |= _suppress_sibling_deltas(values_by_id, hidden, complementary)
changed |= _suppress_dependent_rates(
values_by_id,
units_by_id,
hidden,
suppressed_ids | complementary,
complementary,
)
# Rule 3: the exact-recovery search over same-unit visible figures.
for metric_id in sorted(suppressed_ids):
victim = _recovery_victim(metric_id, values_by_id, units_by_id, hidden)
if victim is None:
continue
complementary.add(victim)
changed = True
return complementary
def suppress_figures(
figures: list[Figure],
*,
threshold: int = SUPPRESSION_THRESHOLD,
complementary_rule: bool = True,
) -> tuple[list[Figure], SuppressionResult]:
"""Apply small-cell suppression to a figure set.
Figures with values in [1, threshold-1] (by magnitude; see
``_is_below_threshold``) are marked as suppressed; true zeros (value = 0) are
preserved. If ``complementary_rule`` is True and a figure is suppressed,
further figures are suppressed until no suppressed value is recoverable from
the ones still visible: any figure in a disclosing signed combination, the
delta of any suppressed period figure, and every percent once any count is
suppressed (see ``_complementary_suppress`` for the three rules and the
module docstring for the disclosure scope).
Returns a tuple of (suppressed_figures, suppression_result). Every field of a
suppressed figure that carries a raw count -- the figure's own value and
display, and its receipt's value, row_count, and slice_hash -- is replaced;
see ``_redact``. This is what makes suppression hold for every artifact that
is later rendered from these figures (the narrative, the charts, the
receipts manifest, and the trace view), not only for a figure's own display.
This function implements the CMS Cell Size Suppression Policy for HHS/HUD
aggregate reporting. Do not alter thresholds or complementary rules without
confirming against the cited primary guidance and recording the change in an
ADR.
"""
# Suppression reads raw values, so it must be given raw figures. An already
# redacted figure has no value to check or to search with, and the threshold
# test would silently skip it: run twice, the second pass would list a
# withheld cell under "unsuppressed" -- a false all-clear on exactly the
# invariant this function exists to assert. Fail closed and say which.
already = sorted(
figure.metric_id for figure in figures if figure.receipt.suppressed or figure.value is None
)
if already:
raise ValueError(
"suppress_figures needs unredacted figures; these are already "
f"suppressed: {', '.join(already)}"
)
suppressed_ids: set[str] = set()
unsuppressed_ids: set[str] = set()
# First pass: identify figures that must be suppressed (magnitude in
# [1, threshold)). True zeros (value = 0) are not suppressed.
for figure in figures:
value = figure.value
if (
value is not None
and figure.receipt.unit == "count"
and _is_below_threshold(value, threshold)
):
suppressed_ids.add(figure.metric_id)
else:
unsuppressed_ids.add(figure.metric_id)
# Second pass: real complementary suppression, driven by arithmetic
# recoverability rather than metric-name substrings.
complementary_ids: set[str] = set()
if complementary_rule and suppressed_ids:
complementary_ids = _complementary_suppress(figures, suppressed_ids)
unsuppressed_ids -= complementary_ids
# Third pass: redact every figure that ended up suppressed, primary or
# complementary, at every field that carries its raw count.
redacted: list[Figure] = []
for figure in figures:
if figure.metric_id in suppressed_ids or figure.metric_id in complementary_ids:
redacted.append(_redact(figure))
else:
redacted.append(figure)
result = SuppressionResult(
suppressed=tuple(sorted(suppressed_ids)),
complementary_suppressed=tuple(sorted(complementary_ids)),
unsuppressed=tuple(sorted(unsuppressed_ids)),
aggregate_only=True,
threshold=threshold,
values=tuple(
(figure.metric_id, figure.value) for figure in figures if figure.value is not None
),
)
return redacted, result
def _figure_is_redacted(figure: Figure) -> bool:
"""True if ``figure`` is already in the redacted state ``_redact`` produces.
Checked against the receipt's ``suppressed`` flag *or* the same
``_REDACTED_DISPLAY`` sentinel ``_redact`` writes, rather than against
``suppress_figures``'s ``suppressed``/``complementary_suppressed`` id sets:
``redact_comparison`` and ``redact_reconciliation`` only ever see the
already-redacted ``Figure`` objects (via ``suppressed_figures``), never those
sets, so these are the signals available to them that agree by construction
with what a reader would actually see rendered. Either signal alone is
sufficient, which is the fail-closed direction: a figure hand-built with one
and not the other is treated as withheld.
"""
return figure.receipt.suppressed or figure.display == _REDACTED_DISPLAY
def _redact_row_direction(row: ComparisonRow, prior: Figure, current: Figure, delta: Figure) -> str:
"""The row's direction, redacted alongside ``prior``/``current``/``delta``.
``direction`` (and the ``arrow`` property derived from it) is computed by
``compute_comparison``/``compute_reconciliation`` from the raw, unredacted
delta -- it asserts a real fact about the sign of that delta ("no change" is
an exact equality claim: prior == current) even when the delta itself, and
the two period figures, render as ``[SUPPRESSED]``. It is not a ``Figure``,
so nothing in ``suppress_figures``'s figure-only search ever sees it or
redacts it on its own. The rule applied here mirrors the delta's own rule
(a delta is redacted whenever either of its periods is; see the module
docstring): if any of this row's three post-redaction figures is in the
redacted state, the direction is too, using the exact sentinel a redacted
``Figure`` displays, so a row never shows three ``[SUPPRESSED]`` cells beside
a still-informative word. A row with nothing suppressed keeps its real,
computed direction untouched.
"""
if any(_figure_is_redacted(figure) for figure in (prior, current, delta)):
return _REDACTED_DISPLAY
return row.direction
def redact_comparison(
comparison: ComparisonResult, suppressed_figures: Sequence[Figure]
) -> ComparisonResult:
"""Rebuild a ``ComparisonResult`` from an already-suppressed figure set.
``compute_comparison`` returns its own copies of the period and delta
figures, embedded in ``ComparisonResult.rows`` and ``ComparisonResult.figures``,
independent of whatever flat figure list a caller later runs through
``suppress_figures``. Left alone, ``render_comparison_table`` would render
those original, unredacted figures -- so a quarter's count or a delta could
leak through the comparison table even though the identical metric_id was
correctly redacted everywhere else. This looks up each row's prior, current,
and delta figure (and every figure in the flat list) by metric_id in
``suppressed_figures``, so the comparison table renders the same redaction
the report, manifest, and trace view do; a caller applies this once, right
after ``suppress_figures``, before any rendering happens.
A row's ``direction`` (and its derived ``arrow``) is rebuilt too, redacted
to the same sentinel whenever ``prior``, ``current``, or ``delta`` ended up
redacted: ``direction`` is computed from the raw delta value, not from a
``Figure``, so it is outside the figure-only search in ``suppress_figures``
and would otherwise survive as a real word -- "no change", exactly -- beside
three ``[SUPPRESSED]`` cells. See ``_redact_row_direction``.
"""
by_id = {figure.metric_id: figure for figure in suppressed_figures}
rows = []
for row in comparison.rows:
prior = by_id.get(row.prior.metric_id, row.prior)
current = by_id.get(row.current.metric_id, row.current)
delta = by_id.get(row.delta.metric_id, row.delta)
rows.append(
replace(
row,
prior=prior,
current=current,
delta=delta,
direction=_redact_row_direction(row, prior, current, delta),
)
)
redacted_figures = tuple(by_id.get(figure.metric_id, figure) for figure in comparison.figures)
return replace(comparison, rows=tuple(rows), figures=redacted_figures)
def redact_reconciliation(
reconciliation: ReconciliationResult, suppressed_figures: Sequence[Figure]
) -> ReconciliationResult:
"""Rebuild reconciliation rows from the publishable figure set.
Same shape as ``redact_comparison``, applied to both the ``outcome`` and
``financial`` sides of each reconciliation row: each side's ``prior``,
``current``, and ``delta`` are rebuilt from ``suppressed_figures``, and each
side's ``direction``/``arrow`` is redacted alongside them when any of that
side's three figures ended up redacted. See ``_redact_row_direction``.
"""
by_id = {figure.metric_id: figure for figure in suppressed_figures}
def redact_row(row: ComparisonRow) -> ComparisonRow:
prior = by_id.get(row.prior.metric_id, row.prior)
current = by_id.get(row.current.metric_id, row.current)
delta = by_id.get(row.delta.metric_id, row.delta)
return replace(
row,
prior=prior,
current=current,
delta=delta,
direction=_redact_row_direction(row, prior, current, delta),
)
rows = tuple(
replace(
row,
outcome=redact_row(row.outcome),
financial=redact_row(row.financial),
)
for row in reconciliation.rows
)
figures = tuple(by_id.get(figure.metric_id, figure) for figure in reconciliation.figures)
return replace(reconciliation, rows=rows, figures=figures)
def filter_for_aggregate_only(figures: list[Figure]) -> list[Figure]:
"""Filter a figure set to aggregate-only (never emit row-level data).
The export boundary accepts only ``Figure`` values: scalar aggregates plus
their receipts. Raw input rows are held inside the compute path and are not
representable in report, manifest, trace, chart, or bundle renderers. Metric
names are deliberately not inspected; a scalar called ``client_id_count`` is
still aggregate data, while name heuristics cannot prove privacy.
"""
return figures