Skip to content

feed-discovery.md

Latest commit

 

History

History
507 lines (492 loc) · 65.5 KB

feed-discovery.md

File metadata and controls

507 lines (492 loc) · 65.5 KB
Raw

Feed-discovery check against the Mobility Database

Run 2026-08-17. Source: mobilitydatabase.org catalog CSV.

This checks whether the feed URL each agency is tracked on still appears in the Mobility Database, and where it doesn't, proposes the catalog feed that looks like the same agency. Candidates are suggestions to verify by hand, not automatic edits.

  • 5 agencies look replaced: the catalog lists a different download URL for the same agency.
  • 2 agencies (KTMB, City of Wasco) surfaced a candidate that a human reviewer rejected — see below.
  • 0 agencies have no catalog match on name or URL.
  • 459 agencies are still on their listed URL: the link is canonical, so any staleness is at the source, not a wrong URL here.

Likely replaced — verify and update the registry

Azienda Napoletana Mobilità (ANM) (anm-napoli)

Münchner Verkehrs- und Tarifverbund (MVV) (mvv-munchen)

Rockford Mass Transit District (rockford-mass-transit-district)

SamTrans (samtrans)

STCP (Sociedade de Transportes Colectivos do Porto) (stcp-porto)

Reviewed and rejected — no valid candidate

Keretapi Tanah Melayu (KTMB) (ktmb-national)

  • Tracked URL: https://api.data.gov.my/gtfs-static/ktmb — not present in the catalog CSV, but confirmed live by hand (HTTP 200, Last-Modified within the last day). Left unchanged.
  • The catalog-matching step returned five candidates for this agency, none of which is Keretapi Tanah Melayu: Rocky Mountain National Park Shuttles (mdb 176, Colorado), a National Park Service scenic railroad (mdb 405, Ohio), Morton's Coaches (mdb 969, Ireland), and two German regional-rail consortium feeds (mdb 1081, mdb 1094). Root cause: _name_tokens() in pipeline/src/scorecard_pipeline/mobilitydb.py tokenizes "Keretapi Tanah Melayu (KTMB)" and "ktmb-national" down to {keretapi, tanah, melayu, ktmb, national}, and "national" was not a stopword — so any catalog feed whose provider or name also contained the word "national" scored as a one-token match, regardless of country or mode. Fixed in the same change that reverts this URL: national added to _NAME_STOPWORDS alongside the already-excluded regional/metropolitan/municipal, with a regression test (test_find_replacements_ignores_generic_national_token in pipeline/tests/test_mobilitydb.py) reproducing this exact pairing. Before the fix, the top-ranked candidate (mdb 176) had been applied to the registry on the branch this report first shipped on, pairing Malaysia's national railway with a Colorado park-shuttle feed; that edit was reverted by hand before merge.

City of Wasco (city-of-wasco)

  • Tracked URL: https://gtfs.dds.dot.ca.gov/gtfs_files/WascoDialaRideFlex.zip — pinned via mdb_id: 1788, so the matcher's highest-trust path fired correctly; the catalog really does list a different URL for this exact mdb id. Left unchanged anyway.
  • Candidate (mdb 1788, City of Wasco): https://gtfs.calitp.org/production/WascoDialaRideFlex.zip
  • This one isn't a matcher bug: registry/us/ca.yaml's city-of-wasco entry carries a hand-written operating_note ("The official DDS ZIP is stale and noncanonical as packaged...") and a license_note that deliberately cites the Caltrans DDS index, not the calitp.org mirror, as the attributed source — a prior curatorial decision to source from the state's own index despite its worse packaging, pinned down by pipeline/tests/test_agencies.py::test_repo_registry_tracks_calitp_hosting_migration. Applying the catalog's pinned-id candidate would have silently reverted that decision. A pinned mdb_id match is correct evidence that a URL moved; it isn't evidence that the moved-to URL is the one this repo wants to track. Worth a follow-up: teach discover to skip (or flag instead of list under "Likely replaced") any agency with a operating_note/license_note that already names the current host, so this doesn't require a manual reject every week the report reruns.

Still on the listed URL — staleness is at the source

The Mobility Database lists the same download URL we already track, so there is no newer canonical feed to switch to. A feed here that is also expired means the agency or its vendor stopped refreshing the export, not that the link moved.