Date: 2026-08-01
Scope: synthetic structural evaluator, Directus source canary, and one CiviCRM target-roundtrip canary
Exit drills can reduce vendor lock-in and improve the bargaining power of organizations serving people with limited alternatives. The same evidence could also be used coercively against vendors or to justify a risky migration.
Real CRM and case-management exports can contain highly sensitive personal, health, benefits, housing, legal, financial, and relationship data. The current release permits synthetic data only. Before production-derived work, the project needs:
- local encrypted ephemeral workspaces;
- least-privilege operator authorization;
- explicit retention and deletion;
- safe logs and crash recovery;
- incident response;
- receipt disclosure review; and
- a documented data-processing basis.
Raw values, IDs, narratives, and attachment bytes never enter receipts. Aggregate counts and hashes can still be identifying in small populations. Capture and normalized bundles do contain record-level values, so the committed Directus canary remains invented-only and the one-command acceptance checks that raw fixture sentinels do not enter receipts, reports, or CLI evidence.
The CiviCRM native capture likewise contains only invented record-level API envelopes and attachment bytes. Its aggregate target result excludes identities, values, content, credentials, paths, and HTTP bodies. The live target harness is restricted to a fresh no-egress local sandbox and remains prohibited for real, customer, employer, client, or production-derived data.
Organizations with the weakest budgets often have the most harmful lock-in. A high-cost enterprise assurance product could widen that gap. Potential models include foundation-funded shared adapters, nonprofit networks, public procurement packs, and an open local runner.
Every receipt carries explicit limitations and separate dimension results. There is no composite portability score. Missing denominators remain indeterminate. The Directus fixture pins its source release, container digest, capture surfaces, schema, license posture, manifest, and mapping decisions. It is labeled one synthetic source-process lab, not a production migration, Directus-wide result, or nonprofit-domain validation.
The CiviCRM evidence is independently bounded: five target-interface probes are
reported separately from the unchanged five-dimension evaluator. All five can
pass while the structural result fails with six missing signals. Target-created
roles, fields, users, ACL groups, ACL group memberships, ACL roles and rules,
helper records, and activities are counted as scaffolding, not relabeled as
preserved source data. Attachment-byte retrieval is explicitly not presented as
case-level attachment authorization. One separate server-rendered Contact
Summary observation remains distinct from one isolated Dashboard → Manage Case
browser observation. The browser result discloses two exact known non-fatal
jquery_notify_unavailable errors, retains no browser artifacts, and makes no
general usability or unobserved-workflow claim. A fourth result reports one
sanitized automated accessibility scan, including two serious findings, while
explicitly withholding any WCAG conformance claim and naming the keyboard,
screen-reader, focus, and zoom/reflow work that automation cannot replace.
A fifth result makes one programmatic keyboard path visible—including the 69
Tab presses needed to reach the Roles disclosure—without labeling that path a
pass or general keyboard-accessibility result.
A sixth result verifies that one target-generated Open Case activity can be
viewed read-only, while keeping that scaffolding separate from the two missing
source audit-history records.
A seventh result verifies one read-only dashboard-to-Contact-Summary path and
the Cases affordance while explicitly withholding contact-editing,
case-navigation, accessibility, and broader-usability claims.
An eighth result follows the target-generated case client through Contact
Summary and Cases back into Manage Case. It keeps the helper classified as
target scaffolding and explicitly withholds source case-client equivalence,
editing, accessibility, and broader-usability claims.
A ninth result records one authenticated deny-principal browser redirect and
protected-content absence while explicitly withholding universal UI/API
authorization and principal-equivalence claims.
A tenth result supplies the same-object positive control: the distinct allow
principal renders the protected Contact Summary directly. It does not convert
the two bounded observations into a general authorization verdict.
An eleventh result observes both synthetic cases through Case Summary and then
records HTTP 500 from one exact-subject filter submission. It exposes a bounded
defect without claiming root cause or general search behavior.
The accompanying evidence-index.json catalogs the normalized export and all
eleven result families without adding a status, score, pass count, or composite
assessment. Its entries retain independent decision scopes, so proximity in the
index cannot be read as combined evidence. Per-entry byte lengths and SHA-256
digests bind the generated set for internal consistency but do not authenticate
the operator or prove any observation true.
The companion CLI verifier checks those bindings, the packaged index and result
schemas, the normalized export contract, and its declared attachment bytes. It
returns no composite result and cannot substitute for the artifact-specific
limitations, structural evaluation, or evidence authentication.
Its closed verification-result contract repeats those boundaries in ordered
machine-readable limitations rather than leaving them only in surrounding
prose.
The current evaluator authenticates no operator, baseline owner, vendor, or timestamp. A future signature can establish issuer and integrity, not truth or completeness. Payer, adapter version, target build, assurance tier, and any human review must be disclosed in production.
The target manifest's image, isolation, automation, and identity statements are also unsigned operator assertions. Hashes make the frozen bundle internally checkable; they do not authenticate who executed the lab or whether the stated pre-write conditions were true.