Skip to content

roadmap.md

Latest commit

 

History

History
108 lines (88 loc) · 5.75 KB

roadmap.md

File metadata and controls

108 lines (88 loc) · 5.75 KB
Raw

Roadmap

Planned direction for tods-validate. Dates are intentions, not promises; items move earlier when users ask for them. Feedback and feature requests are welcome as GitHub issues.

v0.2.0 — Suppression and reporting (shipped 2026-06-12)

  • --ignore TODS-Wxxx (repeatable) and a tods-validate.toml config file so agencies can encode local policy and run the validator in CI without fighting warnings they have decided to accept.
  • --format markdown: a report suitable for pasting into an issue or a working-group thread.
  • Still open: fixes from validating real-world feeds. If you produce or consume TODS and can share a feed (privately is fine), please open an issue.

v0.3.0 — The merge pipeline (shipped 2026-06-12)

  • tods-validate merge feed/ -o supplemented.zip: materialize the "TODS-Supplemented GTFS" the spec describes, so the result can be checked with MobilityData's gtfs-validator. The spec says the merged dataset should form a valid GTFS feed; this makes that property testable.
  • Supplement-internal reference checks (for example, stop_times_supplement.txt:trip_id resolving against supplemented trips).
  • A documented CI recipe chaining merge and gtfs-validator.

v0.4.0 — Distribution and analysis surfaces (shipped 2026-06-20)

  • Docker image on GHCR for CI environments without Python.
  • tods-validate rules (text and JSON, with category and interpretation metadata) and a published JSON Schema for the report format, so dashboards can consume findings without scraping text.
  • A pre-commit hook definition.
  • scripts/benchmark.py for throughput on large synthetic feeds.
  • SARIF and HTML report formats; richer text/Markdown (by-rule grouping, root-cause hints, path-to-green).
  • diff, batch, stats, and anonymize subcommands; a merge manifest.
  • Opt-in coverage (TODS-I50x) and advisory (TODS-I60x) rules via --enable.
  • A public Python API (validate_feed), --baseline, --profile, config extends, and input-safety hardening (SECURITY.md).

v0.5.0 — Spec tracking

  • --spec-version flag. TODS changed substantially between v1 and v2; the validator should be explicit about which spec text it enforces.
  • Validation for spec additions as they are adopted upstream (the spec repository currently has open proposals for rosters, runtimes, and electrification files such as chargers and energy consumption).
  • Offering this project's fixture feeds upstream as a conformance suite.

v1.0.0 — Stability commitments

Gated on the rule set proving out against multiple production feeds and on no rule-ID churn for two consecutive releases. v1.0 means semantic-versioning guarantees on rule IDs, exit codes, and the JSON report schema, plus an acceptance-test corpus in CI.

Out of scope

Validating GTFS itself (use gtfs-validator), GTFS-realtime correlation, and feed editing or repair beyond the merge described above.

Metrics ledger

Per docs/standards/QUALITY-AND-METRICS-STANDARD.md's per-repo Metrics table: project-specific values here, rigor cited to the owning standard. Updated 2026-07-05.

Metric Target Measured by Gate Owner
Line + branch coverage ≥ 90% (published library) pytest --cov --cov-branch in CI AUTO Chelsea Kelly-Reif
Cyclomatic complexity ≤ 10 ruff C901/mccabe AUTO Chelsea Kelly-Reif
SHA-pinned uses: 100% manual + zizmor AUTO Chelsea Kelly-Reif
Workflow SAST findings (High/Critical) 0 zizmor --min-severity high AUTO Chelsea Kelly-Reif
SAST findings (blocking) 0 Semgrep ci --config auto, CodeQL AUTO Chelsea Kelly-Reif
Dependency vulnerabilities (fixable) 0 pip-audit --strict AUTO Chelsea Kelly-Reif
Secrets in tree/history 0 gitleaks (pre-commit + CI) AUTO Chelsea Kelly-Reif
Container CVEs (CRITICAL/HIGH) 0 Trivy in docker.yml AUTO Chelsea Kelly-Reif
Rule ↔ fixture parity 1:1 tests/test_conformance.py AUTO Chelsea Kelly-Reif
Mutation kill-rate (rules engine) ≥ 70% (ratchet; baseline ~65%) mutmut (advisory, weekly) REVIEW Chelsea Kelly-Reif
axe/pa11y violations (HTML report + playground) 0 not yet wired — see docs/CONFORMANCE-GAPS.md#accessibility N/A-not-yet-built Chelsea Kelly-Reif
Perf regression budget ≤ 2x baseline scripts/benchmark.py, not yet a CI gate N/A-not-yet-built Chelsea Kelly-Reif
Screen-reader walkthrough per release not yet committed as an artifact REVIEW-not-yet-built Chelsea Kelly-Reif
Threat model per new surface SECURITY.md, updated ad hoc REVIEW Chelsea Kelly-Reif

Rows marked "not-yet-built" are honest gaps, not silent omissions; see docs/CONFORMANCE-GAPS.md for the open item each maps to.

Release checklist (QM-17)

Run through before creating a GitHub release (tagging triggers pypi-publish.yml, docker.yml, release-corpus.yml, each of which independently re-runs make verify at the tagged commit before publishing):

  1. CHANGELOG.md has a dated section for the version being released (## vX.Y.Z - YYYY-MM-DD), and ## Unreleased items have moved into it.
  2. pyproject.toml version and CITATION.cff version/date-released match the tag you are about to create.
  3. Tag it annotated and signed: git tag -s vX.Y.Z -m "release: vX.Y.Z" (a lightweight or unsigned tag now fails verify.yml's REL-08 check).
  4. Push the tag, then create the GitHub release from it. The three release workflows run automatically; watch that verify (and, downstream, verify-published) succeed before considering the release done.
  5. Confirm the SBOM, provenance attestation, and (for the image) cosign signature are attached/verifiable, per SECURITY.md §Supply chain.