sources.json

{
  "_comment": "Registry of upstream data sources for the OurHike pipeline. Generated/updated by discover_sources.py - re-run that script rather than hand-editing urls here. A source discovery cannot see, because it is not an ArcGIS layer behind the Experience Builder app, is registered by hand and marked with a `kind`: discover_sources.py keeps entries it did not rediscover verbatim, so a hand-written entry survives a re-run. lib/source_registry.py is what reads `kind`, and an entry without one is an ArcGIS feature layer.",
  "photo_licence": {
    "_comment": "The basis on which the Photo1..Photo10 images on the ATC facilities layers may be mirrored and served, recorded rather than assumed - CONTRIBUTING.md, 'A note on data and licences'. This is NOT a Creative Commons grant and must not be rendered as one: ATC holds the copyright, and what is recorded here is the permission to use them, not a licence to sublicense them onward. A club inheriting this project needs to re-confirm it in its own name rather than relying on this line.",
    "author": "Appalachian Trail Conservancy",
    "license": "© ATC, used with permission",
    "basis": "Maintainer authorisation, on the basis of Appalachian Trail Conservancy affiliation",
    "recorded_date": "2026-08-08",
    "open_question": "https://github.com/OurHike/OurHike/issues/98 - ATC's redistribution and attribution terms are unresolved in general, and settling them should replace this line with the agreed terms."
  },
  "atc_licence": {
    "_comment": "The basis on which DATA the Appalachian Trail Conservancy publishes on its own ArcGIS org and services may be reused - photo_licence's sibling, extended from the photos to the data layers by the maintainer's declaration of 2026-08-13 ('anything from the ATC is reusable', restated specifically for the FarOut-measured CSI water distances - #688). Scope, stated tightly: facts ATC itself publishes (half-mile points, the Campsite_Sustainability_Index water distances build_water_distance.py takes, whatever future ATC-org layer a change deliberately adopts). It does NOT cover documents the maintaining clubs publish (per-entry licence fields govern those, e.g. gatc_water_sources below), does NOT override editorial holdbacks that are about safety or partnership rather than licensing (CSI's user-created campsites stay unpublished - SOURCE_SURVEY.md §3b), and is not a grant to sublicense ATC's work onward.",
    "author": "Appalachian Trail Conservancy",
    "license": "© ATC, used with permission",
    "basis": "Maintainer authorisation, on the basis of Appalachian Trail Conservancy affiliation",
    "recorded_date": "2026-08-13",
    "open_question": "ATC's own written terms remain the ideal - SOURCE_SURVEY.md §10's combined ask (capacity, Helene status, CSI, half-mile points) is now a confirmation rather than a blocker, and its answer should replace this line, the same way photo_licence's open question reads."
  },
  "sources": [
    {
      "key": "trail_club_sections",
      "title": "A.T. Trail Club Sections",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/APPA_Trail_Club_Sections/FeatureServer/0",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "centerline",
      "title": "A.T. Centerline",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Centerline/FeatureServer/0",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24",
      "notes": "Also given directly by user as https://arcg.is/1nqL542",
      "blaze_default": "White"
    },
    {
      "key": "side_trails",
      "title": "A.T. Side Trails",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/6",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24",
      "blaze_field": "Blaze"
    },
    {
      "key": "campsites",
      "title": "A.T. Campsites",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/1",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "shelters",
      "title": "A.T. Shelters",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/4",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "parking",
      "title": "A.T. Parking",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/2",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "viewpoints",
      "title": "A.T. Viewpoints",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/5",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "communities",
      "title": "A.T. Communities",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/AT_Communities/FeatureServer/0",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "half_mile_points_from_springer",
      "title": "A.T. Half Mile Points From Springer",
      "provider": "ATC",
      "url": "https://services9.arcgis.com/Nb3RpWJ36xRlYQj2/arcgis/rest/services/Half_Mile_Points040726_NoM/FeatureServer/18",
      "discovered_via": "Experience Builder app c218db84e3ec430db52249797f4cff43 -> Web Map item f0759fe58dcb4c5f9e0f9682a41678cc -> operationalLayers",
      "discovered_date": "2026-07-24"
    },
    {
      "key": "bridges",
      "title": "A.T. Bridges",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/0",
      "discovered_via": "Directly listing the ANST_Facilities FeatureServer root (not in the public Experience Builder map's curated layer list)",
      "discovered_date": "2026-07-25",
      "notes": "Added to registry for later - not yet fetched."
    },
    {
      "key": "privies",
      "title": "A.T. Privies",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/3",
      "discovered_via": "Directly listing the ANST_Facilities FeatureServer root (not in the public Experience Builder map's curated layer list)",
      "discovered_date": "2026-07-25",
      "notes": "Added to registry for later - not yet fetched."
    },
    {
      "key": "at_treadway",
      "title": "A.T. Treadway",
      "provider": "ATC",
      "url": "https://services1.arcgis.com/fBc8EJBxQRMcHlei/arcgis/rest/services/ANST_Facilities/FeatureServer/7",
      "discovered_via": "Directly listing the ANST_Facilities FeatureServer root (not in the public Experience Builder map's curated layer list)",
      "discovered_date": "2026-07-25",
      "notes": "Added to registry for later - not yet fetched. Unclear yet how this differs from the ANST_Centerline layer already in use - worth checking when fetched."
    },
    {
      "key": "atc_trail_updates",
      "title": "A.T. Trail Updates",
      "provider": "ATC",
      "kind": "published_notices",
      "url": "https://appalachiantrail.org/trail-updates/",
      "trust": "authoritative",
      "steward": "Appalachian Trail Conservancy",
      "steward_contact": "https://appalachiantrail.org/contact/",
      "licence": "Facts and a link only, and NOT a grant to mirror ATC's prose. What is published is the mile range, category, states, headline and ATC's own last-updated date, with a link back to their page - facts about the trail, which is a different thing from the paragraphs ATC wrote about them. #458 asked ATC directly what may be republished and was closed without asking, on the maintainer's judgement as an ATC trail volunteer; that decision settles the conservative version and not the broader one, so the body text stays on ATC's own page. A club inheriting this project needs to confirm this in its own name rather than relying on this line - the same footing as `photo_licence` above and the general question in https://github.com/OurHike/OurHike/issues/98.",
      "attribution": "Appalachian Trail Conservancy",
      "reviewed_input": "reference/atc_updates.json",
      "freshness": {
        "kind": "http_etag",
        "url": "https://appalachiantrail.org/trail-updates/feed/",
        "notes": "The RSS feed, used as a change signal and NOT as the content. Measured 2026-08-09: it held 3 items while the page showed 9, so building the artifact from it would silently drop the most consequential closure (features/ATC_TRAIL_UPDATES.md, 'what the feed does not carry'). It does carry a strong ETag that moves when ATC edits, which is all a freshness marker needs. The listing page itself is not usable as the marker: it answers a scripted request with 403 (measured 2026-08-12), so a marker read from it would be UNKNOWN forever."
      },
      "discovered_via": "features/ATC_TRAIL_UPDATES.md - read from ATC's public website, not from an ArcGIS service. Registered by hand because discover_sources.py walks the Experience Builder app and this source is not in it.",
      "discovered_date": "2026-08-12",
      "notes": "Not fetched by fetch_all.py and has no data/raw/<key>.geojson - the build input is the reviewed file named in `reviewed_input`, which a human edits and a merged pull request releases (#460). The freshness marker is what tells that human ATC has published something since they last looked."
    },
    {
      "key": "gatc_water_sources",
      "title": "GATC Water Sources (approach trail + Georgia A.T.)",
      "provider": "GATC",
      "kind": "club_pdf",
      "url": "https://georgia-atclub.org/wp-content/uploads/2026/03/GATC_Water_Sources.pdf",
      "trust": "authoritative",
      "steward": "Georgia Appalachian Trail Club",
      "steward_contact": "https://georgia-atclub.org/contact/",
      "licence": "Unstated - a free public download from GATC's own site, fetched for REVIEW AND CROSS-CHECKS ONLY. Nothing from this source reaches a published artifact until GATC answers a redistribution ask (CONTRIBUTING.md, 'A note on data and licences'; the ask is the same shape as https://github.com/OurHike/OurHike/issues/98 records for opentrail.org). WATER_SOURCES.md §4 sizes this document as 'a pilot-state candidate after an email' and notes its embedded title is 'GATC Water Update July 2020.xlsx', so the 2026-03 file may re-export older data - a second reason the ask precedes any publication.",
      "attribution": "Georgia Appalachian Trail Club",
      "discovered_via": "SOURCE_SURVEY.md §5's club sweep (the 'water-sources PDF (2026-03-02)' row); fetched and parsed live 2026-08-13 (#669). Registered by hand - discover_sources.py walks the Experience Builder app and this source is not in it.",
      "discovered_date": "2026-08-13",
      "notes": "Fetched by fetch_club_pdfs.py to data/raw/club_pdfs/<key>.pdf (never by fetch_all.py), parsed by lib/club_pdfs.py to <key>.json beside it: 65 rows (4 approach trail + 61 A.T., measured on the live PDF 2026-08-13, agreeing with WATER_SOURCES.md §4's count) of GATC mile point + entry text. The PDF's text layer stores each printed row as one string, so the 'Source Name' and 'Distance Off AT' columns arrive fused and are kept that way rather than split by guesswork - lib/club_pdfs.py's docstring holds the measurement."
    },
    {
      "key": "osm_water",
      "title": "OSM water point sources (fourteen A.T. states)",
      "provider": "OpenStreetMap contributors",
      "kind": "geofabrik_extract",
      "url": "https://download.geofabrik.de/north-america/us/",
      "trust": "community",
      "steward": "OpenStreetMap contributors",
      "steward_contact": "https://www.openstreetmap.org/",
      "licence": "ODbL 1.0 - attribution and share-alike, the terms the self-built basemap already complies with; the client's credits screen names OpenStreetMap and each published water point's description carries 'Mapped by OpenStreetMap contributors'. This is the openly licensed half of #529's water answer - no ask pending, unlike the ATC and club rows above.",
      "attribution": "(c) OpenStreetMap contributors",
      "discovered_via": "WATER_SOURCES.md's census for #529 (spike_osm_water_census.py) measured OSM point sources within 250 m of 26% of shelters trail-wide and 40% in the south, against 8% for the shipped supply. Registered by hand - discover_sources.py walks the Experience Builder app and this source is not in it.",
      "discovered_date": "2026-08-13",
      "notes": "Fetched by fetch_osm_water.py to data/raw/osm_water.geojson (never by fetch_all.py): nodes tagged natural=spring, amenity=drinking_water, man_made=water_tap or man_made=water_well across the fourteen Geofabrik state extracts export_basemap.py already downloads - 7,574 nodes on the first full scan (2026-08-13). No freshness entry on purpose: Geofabrik republishes the extracts daily, so a changed-marker would always say yes (export_basemap.py's docstring holds the reasoning; the fetch is skip-if-present with --refetch). export_poi.py folds these into poi_type water at low confidence, dropping each point within 25 m of an opentrail water point as the same node arriving twice."
    },
    {
      "key": "usgs_3dhp",
      "title": "3D Hydrography Program (3DHP)",
      "provider": "USGS",
      "kind": "watched_only",
      "url": "https://3dhp.nationalmap.gov/arcgis/rest/services/usgs_3dhp_all/FeatureServer",
      "trust": "authoritative",
      "steward": "U.S. Geological Survey",
      "steward_contact": "https://www.usgs.gov/3d-hydrography-program",
      "licence": "Public domain, as a work of the United States government — the whole USGS hydrography column (WATER_SOURCES.md §5). Attribution is courtesy rather than obligation, and the client's credits screen already names USGS for NHD.",
      "attribution": "U.S. Geological Survey",
      "freshness": {
        "kind": "arcgis_distinct_values",
        "url": "https://3dhp.nationalmap.gov/arcgis/rest/services/usgs_3dhp_all/FeatureServer/50/query",
        "recorded": "NHD",
        "notes": "The distinct `workunitid` values 3DHP's flowline layer returns for boxes on the trail. `NHD` is 3DHP saying, in its own field, that these lines are the retired dataset republished rather than anything it has surveyed - measured 2026-08-14 at five points spread along the trail from North Carolina (mile 110) to New Hampshire (mile 1,868), every one of them `NHD`. When that stops being the answer, 3DHP has resurveyed the corridor and the migration this pipeline is deliberately not doing becomes worth costing. Changing `recorded` is how a person acknowledges having looked; nothing writes it automatically."
      },
      "discovered_via": "WATER_SOURCES.md §5, following USGS's retirement of the NHD on 2023-10-01. Registered by hand - discover_sources.py walks ATC's Experience Builder app and this is not in it.",
      "discovered_date": "2026-08-14",
      "notes": "Nothing fetches this. It is registered because the pipeline's water derivation (fetch_trail_water.py) reads a dataset USGS no longer maintains, and this is where its replacement will appear. Measured 2026-08-14: for the corridor, 3DHP IS NHD - identical geometry (1,265 flowlines in a box in central Virginia against NHD's 1,265, same vertex count, worst-case separation 0.086 mm) and identical springs (1,796 features, every one within 1 m of an NHD spring) - with the perennial/intermittent FCode collapsed away into an undifferentiated `Channel Line`. Migrating now would trade the only structured seasonality signal the pipeline has for the same geometry, so the entry holds a watch instead of a fetch."
    },
    {
      "key": "usdm_drought",
      "title": "U.S. Drought Monitor weekly class polygons",
      "provider": "National Drought Mitigation Center (UNL), USDA, NOAA, NASA",
      "kind": "weekly_polygons",
      "url": "https://droughtmonitor.unl.edu/data/json/",
      "trust": "authoritative",
      "steward": "National Drought Mitigation Center, University of Nebraska-Lincoln",
      "steward_contact": "DroughtMonitor@unl.edu",
      "licence": "See usdm_licence above - reproduction of the map is explicitly permitted with NDMC's credit line, redistribution of the polygons rides on the maintainer's declaration of 2026-08-15, and the open question is recorded there rather than resolved.",
      "attribution": "The U.S. Drought Monitor is jointly produced by the National Drought Mitigation Center at the University of Nebraska-Lincoln, the United States Department of Agriculture, the National Oceanic and Atmospheric Administration and the National Aeronautics and Space Administration. Map courtesy of NDMC.",
      "discovered_via": "WATER_CONDITIONS.md's survey for #716 measured it against the centerline: 877.1 of 2,172.0 trail miles at least abnormally dry on 2026-08-11, 205.8 of them in severe drought. Registered by hand - discover_sources.py walks ATC's Experience Builder app and this source is not in it.",
      "discovered_date": "2026-08-15",
      "notes": "Fetched by fetch_drought.py to data/raw/drought/usdm_YYYYMMDD.json, the DATED file rather than usdm_current.json: the polygons carry no date inside them, and a weekly product whose artifact cannot name its week leaves the client with only the bake's clock. A USDM week runs Tuesday to Monday and is released on the Thursday, so the fetcher walks back a week at a time - a midweek 404 is 'not published yet', not a failure. export_drought.py clips to a 10 km corridor (27.6 MB national file to about 10 KB gzipped, losslessly - WATER_CONDITIONS.md §4), un-nests the classes so each published band is disjoint, and publishes conditions/drought.json."
    }
  ],
  "usdm_licence": {
    "_comment": "The basis on which the U.S. Drought Monitor's POLYGONS may be republished from OurHike's own bucket, which is a different act from reproducing NDMC's map image. NDMC's permission page (read 2026-08-15) explicitly allows reproducing the MAP given the credit line below, and is silent on redistributing the underlying data - which is what conditions/drought.json does. Recorded on the maintainer's declaration of 2026-08-15, the same footing as atc_licence above: a federally co-produced dataset published for open download is reusable with the required credit. Scope: the weekly USDM class polygons and nothing else. It is NOT a grant to sublicense NDMC's work onward, and it does not extend to any other NDMC or drought.gov product.",
    "author": "National Drought Mitigation Center, USDA, NOAA and NASA",
    "license": "Reproduction permitted with NDMC's required credit; redistribution recorded by maintainer declaration",
    "basis": "Maintainer authorisation, 2026-08-15",
    "recorded_date": "2026-08-15",
    "attribution": "The U.S. Drought Monitor is jointly produced by the National Drought Mitigation Center at the University of Nebraska-Lincoln, the United States Department of Agriculture, the National Oceanic and Atmospheric Administration and the National Aeronautics and Space Administration. Map courtesy of NDMC.",
    "open_question": "NDMC has not been asked directly whether redistributing the polygons is in scope. The wording above is theirs, verbatim from droughtmonitor.unl.edu/About/Permission.aspx; an email to DroughtMonitor@unl.edu would turn this declaration into a confirmation. Until then this is the maintainer's reading, recorded as such rather than as NDMC's."
  }
}