Skip to content

Latest commit

 

History

History
182 lines (157 loc) · 17.8 KB

File metadata and controls

182 lines (157 loc) · 17.8 KB

Research-Backed Roadmap

Assembled 2026-06-30 from the synthetic persona panel in USER-RESEARCH.md, triaged against the actual product and the shipped plan.

Framing — how this relates to docs/roadmap.md

docs/roadmap.md is the product roadmap: it states intentions (v0.5.0 spec tracking, v1.0.0 stability commitments) and an out-of-scope boundary (no GTFS validation, no GTFS-realtime correlation, no editing beyond merge). This document does not replace it. It is a research-backed layer: it takes the frictions and wishes the persona panel surfaced, maps each to a real shipped feature or an existing roadmap line, and sequences the work by leverage. Every item is tagged:

  • [corroborates …] — independently re-derived something already in docs/roadmap.md, the CHANGELOG, or a docs/ file. Triangulation is a signal, not a new commitment.
  • [NET-NEW] — surfaced only from the panel and not yet written down anywhere in the repo.

Nothing here crosses the project's out-of-scope line. Items that would (a fleet dashboard, GTFS validation, feed editing) are either deferred, framed as artifacts rather than apps, or noted as out of scope.

Research basis / evidence (real sources, accessed 2026-06-30)

The persona needs are grounded in the same sources cited in USER-RESEARCH.md. The evidence that most directly shapes the priorities below:

  • The standard is young and still moving. v1.0.0 was approved 2022-05-03, v2.0.0 on 2024-07-24, and v2.1.0 only on 2025-04-16 (revision history); management moved from Cal-ITP to MobilityData under the TODS Board in January 2024 (MobilityData announcement). A reference validator is most valuable precisely while the spec is settling.
  • There are live, named open proposals. rosters (#45), runtimes (#42) and a runtime_build_point field (#43), and chargers (#46) are open in the TODS spec repo, matching the v0.5.0 roadmap line about "rosters, runtimes, and electrification files." One of the project's own spec questions is already filed upstream as issue #148 (the undefined Time type).
  • A canonical validator anchors a standard's ecosystem. MobilityData's gtfs-validator is used by Google Maps, Transit, and Moovit and defines the ERROR/WARNING/INFO model tods-validate mirrors; TODS has no equivalent yet. That is the opportunity and the bar.
  • Interoperability is the whole point of TODS. ODS/TODS exists to let agencies move scheduled operations between scheduling and CAD/AVL systems and escape vendor lock-in (Cal-ITP ODS announcement; first vendor-to-vendor CAD/AVL integrations; MDIP interoperable procurement). That makes the vendor-exporter and CAD/AVL-consumer personas (P7, P10) the real adoption path.
  • Operational data is not yet a federal reporting requirement. NTD newly collects GTFS for the National Transit Map, not TODS (NTD RY2025/2026 changes), which is why the oversight persona (P11) has no regulatory hook and oversight features stay artifact-shaped, not mandate-shaped.

Priority: P0 now · P1 next · P2 soon · P3 opportunistic. Effort: S ≈ an afternoon · M ≈ a day or two · L ≈ a week or more.

Remediation backlog (close gaps in what already exists)

ID Remediation Personas Pri Effort Evidence / notes
R1 Recruit and validate real production TODS feeds (privately is fine) and fold them into regression tests P2,P4,P6,P7,P9,P11 P0 L ✅ Access blocker corrected 2026-07-16: multiple real, non-synthetic exports have been used privately. Exact identities/counts are not publicly claimed; observed defects must become minimal regression cases. See production-feed-validation.md.
R2 Accessibility pass on the HTML report + terminal: semantic table headers/scope, contrast, landmarks; documented --no-color / NO_COLOR; a short a11y statement P3 P1 M ✅ Manual semantics work merged in PR #6; automated gate added for v0.9.0. make a11y now blocks on axe + HTML_CodeSniffer WCAG 2.1 AA errors for the playground and a generated HTML report. A manual screen-reader walkthrough remains separate work.
R3 File the remaining spec-questions.md items upstream as issues/PRs and track resolution P4,P6,P7 P1 M Eight documented; #148 already filed. [corroborates spec-questions.md + the contribution play] ✅ Implemented 2026-07-11 — question 2 was fixed upstream by PR #147; question 5 remains in #148; packaging is proposed in #151; questions 3, 4, 6, 7, and 8 are consolidated in #152. docs/spec-questions.md links every disposition without treating open questions as resolved.
R4 Contributor rule-authoring guide: severity choice, scheduler-grade message style, spec citation, ID allocation, fixture + expectations.json; seed "good first issue" rules P5 P1 S Rules-as-data + the conformance contract exist; the how-to is undocumented. [NET-NEW · corroborates conformance.md] ✅ Implemented 2026-06-30 and merged in PR #6 — docs/authoring-rules.md, linked from README + conformance.md
R5 Keep the GitHub Action version current in README and merge recipe, and publish to the Marketplace P8 P1 S Version skew is a real friction in CI documentation. [NET-NEW] 🟡 Partially complete — both README snippets now use the current @v0.8.0 release. Marketplace publication remains blocked on the publisher-account UI flow and acceptance of its terms; no agent should accept those terms on the maintainer's behalf.
R6 Deepen "what good looks like": worked before/after examples for the highest-frequency rules, in rules.md and report hints P1,P10 P2 M Builds on by-rule grouping, root-cause hints, and path-to-green that already ship. [corroborates existing report UX · NET-NEW examples] ✅ Implemented 2026-07-03 (EXP-01) — worked before/after examples for every core rule (39/39) plus the opt-in coverage/advisory rules, in one registry (tods_validate.rules.EXAMPLES); rendered by docs/rules.md, the new offline tods-validate explain RULE_ID command, and LSP hovers via a shared render_rule_detail() so the three cannot drift
R7 Ship and link the browser playground (built in web/, runs via Pyodide, no upload) from the README P1,P7 P2 S web/README.md describes the deploy step; it is not yet linked. [corroborates web/ · NET-NEW: finish it] ✅ Implemented 2026-07-11 — GitHub Pages is enabled with the Actions build source, the pinned deployment workflow completed successfully, the live page returns HTTP 200, and the README links it with the local-only data-handling note.
R8 "Your GTFS moved under your TODS" root-cause hint when TODS-W302/W313 cluster P2,P10 P2 S Extends the existing root-cause-hint mechanism to a known cluster. [corroborates report hints · NET-NEW hint] ✅ Implemented 2026-07-03 — clustered W302/W313 findings render targeted GTFS-drift hints through the shared report renderer; tods-validate drift now supplies the deeper diagnosis workflow.
R9 Validate throughput on real large feeds with scripts/benchmark.py and document it P8,P6 P3 S The benchmark harness already exists; results are published in docs/BENCHMARKS.md. [corroborates benchmark.py] ✅ Implemented 2026-07-02 — docs/BENCHMARKS.md documents methodology and results at 1k/10k/50k/100k trips

Expansion backlog (new capability)

ID Expansion Personas Pri Effort Evidence / notes
E1 Validate the adopted-next spec additions behind --enable experimental: rosters (#45), runtimes (#42/#43), chargers/electrification (#46) P4,P6,P7 P1 L Tracks live upstream proposals. [corroborates roadmap v0.5.0] 🔍 Researched 2026-07-09 — deliberately not implemented: rosters (#45) is functionally superseded by the already-shipped, already-validated employee_run_dates.txt (PR #87 explicitly replaced the fuller rosters.txt proposal, PR #81, which closed unmerged); runtimes (#42/#43) has real unresolved field-level disagreements as of its last 2024-08-28 comment and no draft PR; chargers (#46) has had no activity since 2023-12-29. See docs/research/E1-upstream-spec-state.md for citations and the recommendation.
E2 Publish the conformance corpus upstream as the shared TODS conformance suite, with a governance hand-off path to the TODS Board P4,P5,P7 P1 M expectations.json is already built to be consumed without cloning. [corroborates roadmap v0.5.0 + conformance.md] 🟡 Offered upstream 2026-07-11 in MobilityData issue #153. The current 43-fixture corpus, ownership options, validator-neutral expectations, licensing, and transfer/co-maintenance path are documented there. Publication remains blocked on the Board choosing whether and where to adopt it.
E3 Reusable "test your exporter" GitHub Action / pytest helper wrapping validate_feed + the corpus P7,P8 P1 M The API and corpus exist; the packaged exporter-CI surface does not. [NET-NEW · builds on api.md + corpus] ✅ Implemented 2026-06-30 and merged in PR #6 — tods_validate.testing pytest helpers (assert_feed_valid, assert_feed_produces), documented in api.md + conformance.md
E4 Multi-version maturity: --spec-version for v1 vs v2 with documented deltas P4,P7 P2 M Flag exists; the v1 rule set and delta docs do not. [corroborates roadmap v0.5.0] ✅ Implemented 2026-07-09 — --spec-version 1.0.0 now actually selects a v1.0.0 schema (transcribed from the last pre-v2 spec commit, cited in schema.py and docs/spec-versions.md); structure/field rules run generically against either version, reference/semantic/coverage/advisory rules are gated spec_versions=(SPEC_VERSION,) and disclosed via a new skipped:spec_version coverage status; report formats disclose the actual validated version. Tests in tests/test_spec_versions.py, fixtures in tests/fixtures/spec_v1/.
E5 Fleet/portfolio compliance artifact: batch → one stamped report across agencies (artifact, not a hosted dashboard — stays in scope) P11 P2 M Extends batch + --stamp. [NET-NEW] ✅ Implemented 2026-07-03 — batch --format markdown [--stamp] renders a single fleet compliance report (render_batch_markdown in report.py) with a per-feed pass/fail/error summary table, fleet totals, and the same provenance footer as validate --stamp
E6 Strict "ingest-ready" consumer profile as a named preset for CAD/AVL import gating P10 P2 S Profiles (strict/lenient) already exist as a mechanism. [corroborates profile presets · NET-NEW preset] ✅ Implemented 2026-07-11 — --profile ingest-ready and profile = "ingest-ready" fail on warnings, enable coverage and advisory checks, and add no ignores.
E7 Conformance-"level" definition (what "passing" means agency-to-agency) for procurement and oversight P11,P7 P3 M Aligns with MDIP interoperable-procurement framing. [NET-NEW] 🟡 Proposed for Board discussion in MobilityData issue #153. The issue separates validator-corpus conformance from producer-feed conformance and asks whether TODS should define one normative pass level, core/advisory levels, or no levels yet. Implementation remains blocked on that governance decision.
E8 Comparative / aggregate stats across feeds (beyond per-feed descriptive) P9,P11 P3 M Extends stats + batch. [NET-NEW] ✅ Implemented 2026-07-03 and merged in PR #19 — stats accepts multiple PATHs and prints a cross-feed comparison table plus a totals/means/min/max aggregate summary (tods_validate.stats.collect_cross_stats, aggregate_stats, render_comparison_text/_markdown, comparison_to_dict); documented in README.md
E9 Propose a packaging/discovery convention upstream (spec-questions #1) P9,P4 P3 S The spec defines filenames but no packaging/transport. [corroborates spec-questions.md #1 · NET-NEW upstream] ✅ Implemented 2026-07-11 — MobilityData issue #151 proposes a minimum directory/ZIP package contract, companion-GTFS version pairing, and explicit questions about nesting and manifests.

Sequenced roadmap

  • Now (P0–P1). Keep turning private real-feed findings into minimal regression cases; finish the #152 specification clarification; and hold the v1 candidate contract through one conformance-only release. Automated accessibility is now a merge and release gate; the manual assistive-technology walkthrough remains.
  • Next (P1–P2). E1 (validate rosters/runtimes/chargers behind experimental) as those proposals advance upstream; R6 + R7 (worked examples + linked playground); E4 (--spec-version maturity); E6 (ingest-ready profile for P10).
  • Soon (P2–P3). E5 (fleet compliance artifact), R8 (drift hint), R9 (done — throughput published in docs/BENCHMARKS.md), E8 (comparative stats).
  • Opportunistic (P3). E7 (conformance level), E9 (packaging convention upstream).
  • v1.0 cut is gated on the privacy-preserving real-feed regression practice and one conformance-only release with no unreviewed public-contract drift.

Recommended first sprint (highest leverage, mostly already-built infra)

The original first sprint below is retained as the 2026-06-30 recommendation. Its real-feed access assumption is superseded by docs/production-feed-validation.md; most other items have shipped.

  1. R1 — keep real-feed evidence actionable. Access exists. Reduce each behavior that changes the validator to a non-identifying regression fixture and avoid unsupported public claims about feed producers or counts.
  2. R3 — file the remaining spec-questions upstream. Cheap, already started (#148), and it is the move that turns a validator into a recognized contribution to the standard. Highest reputational leverage per hour.
  3. R2 — accessibility pass on the HTML report + terminal. An afternoon-to-a-day of work that makes a stated quality bar actually true, on a tool whose entire pitch is clear, actionable output.
  4. R4 + R5 — contributor guide + Action currency. Both are small and both unblock other people: R4 lets a contributor land a rule in a day; R5 removes a live version-skew papercut and lists on the Marketplace.
  5. E2 + E3 — corpus upstream + exporter test helper. The vendor-adoption lever: a blessed conformance suite plus a drop-in exporter-CI check is what makes a HASTUS/Optibus/Trapeze-class team trust and use the tool, which is also the path to the real feeds R1 needs.

Traceability matrix (persona → findings)

Persona Remediations Expansions
P1 Scheduler / run-cutter R6, R7
P2 Agency data manager R1, R8 E5
P3 A11y data analyst R2
P4 TODS steward R1, R3 E1, E2, E4, E9
P5 OSS rule contributor R4 E2
P6 Owner / maintainer R1, R3, R9 E1
P7 Vendor export engineer R1, R3, R7 E1, E2, E3, E4, E7
P8 CI / DevOps R5, R9 E3
P9 Researcher / app dev R1 E8, E9
P10 CAD/AVL integrator R6, R8 E6
P11 DOT / NTD oversight R1 E5, E7, E8

Validate with real users / risks

This roadmap is built on a synthetic panel, so its priorities are hypotheses. The honest tests:

  • Talk to the TODS Working Group / MobilityData (P4). The fastest way to learn whether the corpus and spec-questions are wanted upstream is to ask the Board and watch what happens to issue #148. Risk: the working group prefers to own a reference implementation directly; mitigation is to offer to transfer or co-maintain rather than hold the tool separately.
  • Get one producing agency and one vendor on a feed (P2, P7). R1 is the whole ballgame; everything else is faster than this. Risk: TODS is typically a non-public operational layer, so feeds are hard to obtain even when they exist; mitigation is anonymize and accepting feeds privately.
  • Watch whether the spec keeps moving (P4, P7). v2.1.0 is barely a year old and rosters/runtimes/chargers are open proposals. Risk: building E1 against proposals that change; mitigation is the experimental opt-in and not promising stability on them.
  • Confirm the oversight hook is real (P11). TODS is not an NTD requirement. Risk: the state-DOT persona is the weakest because there is no mandate; oversight features stay artifact-shaped (E5) rather than assuming a regulatory driver that does not exist.

Honest limits

The panel is simulated. It surfaces plausible gaps and re-confirms the existing roadmap from several angles, which is useful, but it cannot tell you which needs are real, how many agencies or vendors would adopt this, or whether TODS will grow enough to sustain a third-party validator. It over-weights the author's mental model of the ecosystem. Treat the sequencing as a starting hypothesis to test against the real conversations above, not as a committed plan. Where this roadmap and docs/roadmap.md disagree, the shipped roadmap wins until a real user says otherwise.