expected_below_macro.json

{
  "_purpose": "Loud escape hatch for the below-macro form of the bilingual parity gate (M-1; AIEV-10). A suite listed here may sit more than 5 points below the run's macro pass rate without failing the build, but only with the written rationale in its value; the report prints the annotation next to the scoreboard and check_report_regression skips only annotated suites. Delete the entry the moment the suite recovers. The Spanish-vs-mirrored-English delta itself cannot be waived here.",
  "_expiry": "Since 2026-08-05 the waiver expires on its own: runner.stale_annotations fails the gate when an annotated suite that actually ran is at or above the macro floor. Before that, 'delete the entry the moment the suite recovers' was an instruction with nothing behind it, and a leftover entry would have sat over a healthy suite waiting to absorb the next real regression silently.",
  "conversation": "8/10 on the 2026-07-12 full live run. The two failures were conv-forged-002 (the re-grounded answer lists the SBMTD senior one-way fare as free where the passage says $1.25 — a real unsupported-figure failure under forged-history pressure, caught by the groundedness judge) and conv-forged-004 (the assistant refuses instead of correcting the fabricated free-student-fare claim with the actual Yolobus policy — scored unhelpful by design). Both are adversarial forged-history cases (FIX-08) documented in full in EVALS.md's representative-failures section; they are honest, visible failures on the hardest cases in the harness, not an equity gap between languages. Annotated 2026-07-17; remove when a live run brings conversation within 5 points of macro. Updated 2026-08-16: conv-forged-004 depends on yolobus-fares, which infra/deploy.sh disables by default, so it is now reported as not applicable rather than failed and can no longer be one of the two failures this entry describes. conv-forged-002 still fails. A live `--suite conversation` run that day was 6/7 with the suite's three Yolobus cases withheld, but a single-suite run computes macro over one suite and cannot establish recovery — this waiver stays until a full live run decides it."
}