This is the plan for turning the scorecard from a thing you look at into a thing
agencies and the people who support them rely on. It builds on
product-roadmap.md (user value over time) and
roadmap.md (infrastructure), and is grounded in what the rest of
the GTFS ecosystem already does, so the scorecard fills a real gap rather than
rebuilding what exists.
California already publishes an official monthly GTFS quality report per agency
(for example, Santa Barbara MTD, June 2025).
It is thorough and free. It is also technical: it lists validator notices like
foreign_key_violation, shows realtime-completeness percentages with no
benchmark, and runs a 24-item pass/fail checklist with no statement of which
item a rider would feel first or what to do about it. A manager at a 20-bus
agency cannot act on it.
Commercial platforms (Optibus/Trillium, Swiftly) monitor GTFS well, but as features of paid products that small and rural agencies do not buy. The canonical GTFS validator and the Mobility Database are the right foundation to build on, not to duplicate.
So the scorecard's job is narrow and unclaimed: the free, plain-language monitoring-and-prep layer on top of the canonical validator, made for the people who support small agencies. Not another validator. Not a creation tool.
- An agency manager visits when something feels wrong, or once after a vendor export. They become a returning user only if the tool tells them something before it bites: "your feed expires in 12 days." An expired feed gets the agency dropped from Google trip planning, which is the concrete, feared consequence that makes monitoring worth opting into.
- A state liaison or customer-success role opens it before a check-in call. They become a returning user if the tool prepares the call: the agency's grade, what changed since last month, the three things to raise.
The service has to serve both as habits, not one-time visits.
Each stage is shippable on its own and checked against the principles in
product-roadmap.md: findings framed as fixes, no
shaming, accessibility first.
Let an agency claim its page and opt into alerts. The expiry warning is the anchor because the consequence is concrete; regression alerts ("your grade fell from B to D this week") and a periodic all-clear round it out. This is what makes an agency return.
- Claim and verify. An agency proves control of its email (double opt-in to
an address at the agency domain, or a token placed in
feed_info). Verification is a hard gate: nothing is emailed to an unverified address. - Opt-in granularity. A subscriber chooses which alert kinds they want (expiry only, or expiry and regression) and which agencies they follow.
- Built so far: the alert digest (
alerts.py); per-subscriber filtering, the verification gate, and per-kind opt-in (notify.py). Live infrastructure: SES sending is verified forgtfsscorecard.organd the digest send path works; the private opt-in store is a DynamoDB table the pipeline reads (scorecard notify --table). What remains is the public self-serve claim/verify endpoint, applied deliberately. See ADR 0004.
The rubric is currently the scorecard's own. Make it a crosswalk to the two authorities this audience is measured against, so the grade is credible rather than invented.
- The California Transit Data Guidelines and Minimum GTFS Guidelines: show the grade alongside the official checklist so a manager sees "scorecard B, and here is where you stand on the state's checklist."
- The MobilityData GTFS Grading Scheme, which covers the qualitative, rider-facing checks (signage and name accuracy, headsigns) that an automated validator cannot catch. The Rider-experience category should align with it explicitly and automate what it can.
Built so far: the crosswalk is written (crosswalk.md) and
rendered as an on-site page at /crosswalk/, and agency pages carry the
California Minimum GTFS Guidelines checklist.
For liaisons and customer-success roles: a cohort dashboard (the rollups exist), plus what a call actually needs — "what changed since last month" per agency, a one-click call-prep export, private notes per agency, and shared-fix detection across a portfolio ("one export setting fixes these five agencies"). This is the feature that puts the supporter audience in the tool daily, and they are the distribution channel to agencies.
Built so far: the weekly portfolio digest for liaisons (scheduled in the
daily workflow, Mondays), the call-prep brief at /agency/<id>/brief/,
shared-fix detection on every program page, and the client-side supporter
workspace: saved cohorts and per-agency private notes, kept in the browser's
own storage so nothing personal ever reaches the pipeline. Custom
MPO-boundary cohorts remain open (RR:E10).
Finding to plain-language fix guide to re-check to "you fixed it, your grade went up." Tailor the guidance to where the data is produced: detect the export tool from the feed and point at its settings. The confirmed fix is what turns a one-time fixer into an advocate.
Finish realtime scoring (scoped; gated on keyless or key-managed endpoints), and
flag the RY2026 NTD requirement to provide a stable agency_id for every
represented reporter and crosswalk it to the NTD ID on P-50. The feed value does
not have to equal the five-digit NTD ID.
Rural agencies are struggling with it, it is federal, and it lives in a GTFS
field — so it gives agencies a compliance reason to care, not only a quality one.
Built so far: the agency_id presence pillar and equality flag. Presence is
checked for every US feed and affects only the NTD-readiness status. When an
agency's five-digit NTD ID is on file (ntd_id in the registry), the scorecard
also compares it with the feed value. A difference is allowed, carries no score,
and points the reader to the required P-50 crosswalk. See
ADR 0016.
Per-state guideline profiles so the rubric cites the right authority for each state, and a partnership with National RTAP, which already hosts rural agencies' GTFS for free and runs weekly support sessions — both a data source and a channel to exactly these users nationwide.
These are adoption blockers for public agencies, not afterthoughts.
- Claim and verify before any alert; honor delisting requests (see
listing-policy.md). - Keep the "data-quality lens, not a compliance determination" framing, and never publish a bare failing grade without the fix and the context next to it.
- Accessibility stays visibly first-class (WCAG 2.2 AAA).
The public site stays static. Stage 1, now live, adds a small verify function and a DynamoDB subscription store the daily run reads; everything else (scoring, rollups, static pages) stays in the daily batch. The discipline holds: single-digit dollars a month until something is genuinely relied on. A fuller agency datastore, beyond opt-in subscriptions, waits until claimed agencies and notes demand it.
Not by agencies finding it, but by the people who support them bringing it: state data programs, regional associations, the MobilityData community, and RTAP's weekly sessions. Plus badge backlinks from agencies proud of a good grade, and filling the actionability gap the official monthly report leaves.
This is public-interest infrastructure, not a venture product. The agency-facing scorecard stays free. Realistic support: operating it under a state data program or with National RTAP; federal technical-assistance and innovation funding (FTA 5311(b)(3) RTAP, FTA 5312); and open-source contribution with sponsored hosting. A paid tier, if any, is for the consultancies and programs that manage many agencies (the Stage 3 workspace), never for the agencies themselves.
- Claim, verify, and expiry-alert opt-in (Stage 1).
- The California Guidelines crosswalk on each scorecard (Stage 2).
- One supporter cohort workspace with "what changed" and a call-prep export (Stage 3).
- A claim/delisting policy page and an accessibility audit, so it is safe to show real agencies their grades.
This is the smallest sequence that makes an agency come back and a supporter open it before a call.