GTFS Scorecard

Is your transit agency's GTFS feed any good? A plain-language quality grade, the three things to fix, and why each one matters to riders.

A data quality scorecard for small transit agencies. It fetches an agency's GTFS Schedule feed and, when one is published and configured, its GTFS-Realtime feeds; runs the canonical MobilityData validator, and turns the results into a letter grade with a short list of concrete fixes, written for the transit manager who inherited the feed from a vendor, not for developers.

Pilot agencies: Unitrans (ASUCD / City of Davis) and Yolobus (Yolo County Transportation District). Beyond the pilots, the registry contains more than 1,400 curated feed records, mostly in the United States and Canada, plus reviewed canaries across Europe, Asia-Pacific, and South America. The repository keeps the generated corpus registry-bounded, with more than 1,100 numeric scorecards published. Scoring is scheduled daily, and the public status page reports the exact current configured and published counts along with when the latest run completed. A feed record is not always a distinct transit agency: regional feeds, modal variants, and retired aliases are counted separately while the identity registry is reconciled.

Live: gtfsscorecard.org — with the latest completed-run evidence at gtfsscorecard.org/status.

Status: Beta. The three schedule categories score for every published scorecard. Realtime quality is scored only when a usable realtime feed is configured and measured; otherwise it is shown neutrally as not yet measured. Any agency can be added through registry/intake.yaml.

What an agency gets

• An overall grade from the categories that can be measured: correctness, freshness, rider experience completeness, and, when available, realtime quality.
• "Top 3 things to fix", in plain language with effort hints. Findings are framed as fixes, never as failures.
• For U.S. agencies only, an NTD GTFS-readiness read (published, valid, current, and agency_id present) plus a neutral check of whether agency_id happens to equal the five-digit NTD ID. Equality is optional; RY2026 presence and the P-50 crosswalk are not.
• Trend history, one JSON artifact per agency per day.
• An embeddable grade badge (<agency>/badge.svg) the agency can put on its own developer page.

The scoring methodology, with citations to the California Transit Data Guidelines v4.0 and the validator's rule taxonomy, is in docs/rubric.md. Methodology changes are governed: a validator-version bump must attach the shadow-run impact report from scorecard canary before it lands (rubric.md, "Governed upgrades"). Feed sources and licenses are in docs/feeds.md. Forward planning is split in two: the infrastructure and scaling plan is in docs/roadmap.md, and what the product becomes for its users is in docs/product-roadmap.md.

What's on the site

Each agency gets a scorecard page, plus three companion pages written for the different seats at an agency check-in: a board one-pager (/agency/<id>/board/), a call-prep brief (/brief/), and, when comparable finding-clearance records exist, a clearance log (/fixes/). Around those sit:

• Coverage views — the coverage overview (/pulse/), most common problems (/problems/), realtime reliability (/realtime/), optional-feature adoption and accessibility data coverage (/adoption/), a consumer feature finder with translation-language and accessibility filters (/app/#/?view=features), and worldwide agency and route maps (/map/, /routes/). U.S.-specific NTD readiness (/ntd/) and equity (/equity/) stay available as regional modules.
• Program pages (/program/<state>/) for 46 states plus DC and named cohorts from rollups.yaml, each with the fixes shared across the group.
• Practitioner tools — request a one-off score through GitHub or run it locally (/try.html), check a feed before publishing (/check/), compare two agencies (/compare/), query the dataset with SQL in the browser (/query/), and put feed quality in a vendor contract (/procurement/).
• Board-ready outputs — each agency has a printable one-pager; see the live Unitrans example, or generate a self-contained, custom-branded offline report with the board-report tool.
• A fix knowledge base (/fix/<rule>/, one plain-language page per common validator finding) and the standards crosswalk (/crosswalk/).
• Machine-readable surfaces — the versioned read API (docs/api.md), a Parquet table for bulk SQL, per-agency badges and conformance marks, Atom change feeds, monthly citable dataset releases, and a read-only MCP server (docs/mcp.md).

The country controls describe only the current covered set. They are not a census of a country or region. The live evidence and readiness result for a bounded European GTFS beta are on the status page; the release gate and scope limits are in docs/global-expansion.md.

Quickstart

Requires Python 3.12+, uv, and Java 17+ (the validator jar is downloaded automatically on first run).

cd pipeline
uv sync
uv run scorecard run --all

This fetches today's snapshot of every configured feed, validates and scores it, and writes artifacts to data/artifacts/<agency>/<date>.json plus a latest.json and a cross-agency index.json. Re-running a day is idempotent. Checks (from the repo root; mirrors the CI gate):

make verify

Run the web app locally

The frontend reads the JSON artifacts over HTTP. Serve the repo root and open the page through http://, not by double-clicking the file:

cd ..            # repo root, so data/artifacts/ is reachable
python3 -m http.server 8000
# then open http://localhost:8000/web/index.html

Opening web/index.html as a file:// URL leaves the page stuck on "Loading scorecards…": browsers block ES module loading and fetch over file://, so the app never runs. Any static server works; the only requirement is that data/artifacts/ sits one level above web/, which the ../data/artifacts fallback in web/src/app.js expects.

Use it in CI

Gate your own pipeline on feed quality with the published action. It scores the feed, prints the grade and the top fixes in the job log, and fails the build if the feed drops below min-grade or expires within min-days-to-expiry:

- uses: ChelseaKR/gtfs-scorecard@v1
  with:
    feed-url: https://your-agency.example/google_transit.zip
    country: CA
    min-grade: C
    min-days-to-expiry: 14

country is the feed's assigned ISO 3166-1 alpha-2 code; it defaults to US for existing workflows. Both thresholds are optional; leave one blank to skip that check. Full input reference and a complete workflow are in docs/ci-action.md.

Operating at scale

Beyond run, the CLI carries the commands the rollout plan needs:

scorecard sync --country US --state California   # propose keyless, untracked
                                                 # Mobility Database entries
scorecard shards --count 4                        # JSON fan-out plan for CI
scorecard reindex                                 # rebuild index.json from disk
scorecard rollups                                 # publish program rollup artifacts
scorecard render-site                             # crawlable static pages + sitemap
scorecard alerts --out digest.md                  # expiry/regression digest
scorecard notify                                  # per-subscriber digest (dry run)

notify builds a feed-health email for each opt-in subscriber in subscriptions.yaml, containing only the agencies they follow and only when one needs attention. It prints the emails by default; the daily workflow sends them via SES once an operator verifies a sender, applies the SES grant in infra/artifacts, and sets the SES_FROM repo variable.

The daily workflow fans agencies out across a parallel matrix and can mirror artifacts to a CloudFront-backed S3 bucket once infra/artifacts is applied (ADR 0002). Program rollups are configured in rollups.yaml as named cohorts (a liaison's own agencies, a district, the whole state) and shown at #/programs; an agency "needs attention" when its feed is expiring or its grade regressed, not merely when it scores below a B. The published JSON is a documented read API (docs/api.md); a flat catalog of every agency (grade, score, feed URL, days-to-expiry, top fix) is served at /catalog.json and /catalog.csv so a consumer needs one request, not one per agency.

Roadmap status: built vs deployed

The roadmap plans the path from two pilot feeds to a worldwide-capable service. The Year 1 software is built, tested, and mostly deployed; the deploy runbook walks through the AWS stacks and carries the current deployment status.

Roadmap piece	In the repo	State
Mobility Database sync	scorecard sync (mobilitydb.py)	run on demand
Scheduled scoring run	scorecard shards + CI matrix	daily schedule live in Actions; observed completion is on /status/
Expiry/regression alerts	scorecard alerts, notify --send, infra/alerts	applied and live; SES sender verified, digest sends daily
Artifacts on S3 + CloudFront	infra/artifacts	applied and live; daily mirror on, site still serves from Pages
Self-serve submission form	web/submit.html, infra/submit	applied and live; submissions open reviewable pull requests
Instant scoring	web/try.html, infra/instant-score	built; falls back to the issue-form path until applied (ADR 0029)
Fan-out compute (Year 2)	infra/compute (SQS + worker)	built; apply when the daily run outgrows the Actions matrix

The cohort drafted from the Mobility Database has grown well past the first California pass: the manifest-backed registry now carries more than 1,400 curated feed records, mostly across the US and Canada, with a 148-record reviewed European cohort across 17 countries. It now includes a geographically diverse reviewed canary cohort, scored daily (a 2026-07 dedupe pass removed ~350 records that duplicated an already-listed feed).

Add your agency

Any agency with a public GTFS feed can be added with one YAML block in registry/intake.yaml and a pull request; the walkthrough is docs/add-your-agency.md. The live web form at web/submit.html does the same without YAML through the deployed infra/submit endpoint; every submission still opens a pull request for human review before publication.

Support

The public scorecards, data, API, local pre-publish check, and request-backed one-off scoring remain free. The support page explains the project's infrastructure needs and accepts sponsorship inquiries; it does not advertise a payment rail that is not ready.

Standards conformance

This repo is governed by the shared portfolio standards vendored at docs/standards/ (pinned to a tag — never edited locally; see the integrity note in that directory's history). Per docs/standards/README.md's conformance rule, every applicable standard is declared here; none is silently skipped.

Standard	Applies?
Code Quality	Applies (Python; TS/Node N/A — web/ is no-build vanilla JS)
Security & Supply-Chain	Applies (ASVS L1 shape: no auth, no PII store)
CI/CD	Applies
Observability	Applies — Tier B (frontend) + Tier C (batch pipeline); see ADR 0031
Accessibility	Applies fully — civic content, self-declared WCAG 2.2 AAA (see docs/accessibility.md, docs/vpat.md)
Internationalization	Applies — civic transit data, public-facing; exemption path unavailable
AI Evaluation	N/A — no LLM/model component: no model inference in any user-facing or decision-making path (AI-EVALUATION-STANDARD §0); the MCP server (server.json) is read-only data retrieval, no LLM SDK. Flips to APPLIES on first LLM SDK use.
Quality & Metrics	Applies (data-quality/lineage named for this repo explicitly)
Documentation	Applies
Release & Versioning	Applies — reusable Action tags (v1/v1.3.0), monthly dataset releases, MCP registry entry
Responsible-Tech Framework	Applies (audits A-F; AI-governance rows N/A — no AI system)

Open gaps per standard, as of the most recent conformance audit, are tracked in docs/standards-conformance-gaps.md rather than restated here.

Observability

Tier B (frontend) + Tier C (batch pipeline) — not the OBSERVABILITY-STANDARD's default Tier-A mapping for this repo's class; see ADR 0031 for why. In short: the deployed system is a scheduled Actions batch job plus a static site, not a long-lived hosted service (infra/compute, the piece that would make this Tier A, is built but not applied). Tier C's health/tracing/SLO controls are N/A (no network surface to probe); Tier B's Core Web Vitals lab gate applies and is tracked as open work (Lighthouse currently asserts accessibility only). Real-user-monitoring beacons are declined by design — see the ADR.

Versioning

SemVer. The public API surface is the published JSON schema (schema_version), the GitHub Action's inputs (action.yml), and the CLI (docs/api.md). pipeline/pyproject.toml's [project].version is the single source of truth; CITATION.cff and server.json mirror it, and pipeline/scripts/check_versions.py fails make verify if they drift. Supported-version policy: latest major only — this is a single-deployment civic tool, not a library with multiple consumers pinned to old majors.

This repo also produces releases: the marketplace GitHub Action (tagged v1), monthly citable dataset releases (dataset-release.yml), and an MCP registry entry (server.json).

Run your own instance

A state DOT, national RTAP, or country program can fork this repo and stand up its own branded instance — own domain, own agency registry, own organization name — from config and a deploy, no code change: docs/fork-quickstart.md.

Layout

registry/       manifest-backed agency shards; add yours to intake.yaml
instance.example.yaml   fork branding template; copy to instance.yaml to rebrand
rollups.yaml    program rollups (portfolio views across many agencies)
pipeline/       Python pipeline: fetch -> validate -> score -> publish
web/            static frontend; reads only published JSON
infra/          Terraform: artifacts CDN, submission function, fan-out compute
docs/           feeds, rubric, roadmap, api, fixes/ (KB), decisions/ (ADRs)
data/           raw snapshots (ignored) and published artifacts (committed)

For Claude Code

CLAUDE.md is the build spec: product framing, rubric, and quality bar. The original four build phases have shipped; current direction lives in the roadmaps and docs/ideation/. Hard rules: every metric ships with its plain-language explanation; accessibility (WCAG 2.2 AAA) is non-negotiable in the web app; agencies without realtime are shown neutrally, never shamed.

Maintainer

Built and maintained by Chelsea Kelly-Reif, starting from the transit systems in Davis, California. Contributions and corrections are welcome; agencies can ask to be added or removed under the listing and removal policy.