Skip to content

ROADMAP.md

Latest commit

 

History

History
85 lines (67 loc) · 17.7 KB

ROADMAP.md

File metadata and controls

85 lines (67 loc) · 17.7 KB
Raw

OurHike — Roadmap / Build Plan (Draft v1)

Companion to OurHikeValues.md, FEATURES.md, and TECHNICAL_ARCHITECTURE.md. Ordered, phased list of what needs to get built and how, from where we are now (decided architecture) to a shippable v1. Expect this to shift as decisions firm up — check items off as they're done, add detail as it emerges.

Feature design docs

Ten features have full design drafts, written 2026-07-28 and now organized in features/ so nothing gets lost. Each is already linked in context from FEATURES.md and from the phase items above — this section is just the one place that lists all ten together:

  • features/AUTHENTICATION.md — Google/Apple/email sign-in, verification, optional MFA. Post-MVP, but build first — Segments, Volunteering, Report a Problem, and Map Options' closures all depend on it.
  • features/TRAIL_BLAZE_COLORS.md — render the trail line in its real painted blaze color, with a neutral fallback when unknown. A v1 MVP correctness detail, not Post-MVP.
  • features/SEGMENTS.md — hierarchical Hike → Segment tree so hikers can plan and track a thru-hike, section, or day-hike however suits them. Post-MVP (Extras).
  • features/TRIP_PLANNING.md — builds on Segments: day-hiker waypoint planning, bulk multi-day date shifts, POI-aware planning assistance. Post-MVP (Extras).
  • features/REPORT_A_PROBLEM.md — hiker-submitted condition reports (blow downs, trash, bad hikers, flooding, shelter repair, animals), with "bad hikers" routed internal-only. Post-MVP (Community reporting).
  • features/VOLUNTEERING.md — club work-project management, plus upcoming projects shown on the map to encourage hikers to join one. Post-MVP (Trail magic).
  • features/MAP_OPTIONS.md — user-controlled background tile source, roads/sidewalk-walkability overlay, DuckDB-backed snap-to-trail for Segments, and trail-closure marking/display. Mostly Post-MVP; its map-chrome section (legend/scale/locate/zoom) is a v1 MVP detail spec.
  • features/HIKER_SAFETY.md — moderator-escalated serious warning pins, a configurable anonymity window for comments, a responsibly-sourced NWS weather-alert relay and elevation-aware daily conditions, and a deliberately conservative wrong-way/off-trail alert scoped as the only notification OurHike ever sends. Post-MVP.
  • features/DATA_NUDGES.md — low-friction, non-gamified prompts to keep water/shelter/resupply data fresh: no notification of any kind, just visual map prominence for stale POIs and open reports, self-limiting the moment anyone contributes. Post-MVP (Community reporting).
  • features/UX_CUSTOMIZATION.md — light/dark mode, metric units, which waypoints display and at what detail, and auto-rotate/compass. Splits persistent settings from on-map controls up front; auto-rotate is real Post-MVP work given the platform constraints, the rest is mostly MVP-detail or light Post-MVP settings polish.

Phase 1 — Data pipeline

  • DuckDB spatial spike. Prove the core operation works before building on top of it: buffer the AT centerline by 30 miles, clip real data against it, export the result. (Done — see pipeline/spike_corridor.py. Buffered the full 3,025-segment centerline by 30mi, unioned into one valid corridor polygon (~81,138 sq mi), clipped real ATC campsite/shelter data against it — all 232 campsites and 280 shelters fell within the corridor, as expected for official on-trail sites. Used ATC data only, not OSM — see note below on rethinking OSM's role first. Hit and documented an ST_Transform axis-order gotcha in pipeline/README.md, worth knowing before anyone else touches that call.)
  • Think through open-source trail data strategy more deliberately before building more ingestion. Prompted by realizing the OSM pull was about to become "grab roads because the architecture doc says so" rather than solving a real hiker need — and by USGS's National Digital Trails page on the many ways to access their trails dataset. Worth deciding deliberately: what background/reference map layer to use (raster vs. vector, and from where — USGS, OSM, something else), and which specific gaps in ATC's own data (confirmed gap: no water-source or general resupply layer) get filled by which external source. Deserves real thought, not an MVP-speed decision.
  • Ingestion scripts — pull raw extracts from ATC (POIs), USGS (elevation/topo), OSM (roads/land cover) into formats DuckDB reads directly (GeoJSON/Shapefile/GeoPackage/FlatGeobuf). (ATC side started — see pipeline/: discover_sources.py finds layer URLs from ATC's public ArcGIS map, fetch_all.py pulls all 9 registered layers. USGS/OSM ingestion not started.)
  • Elevation data, moved into MVP 2026-07-28 (see FEATURES.md's "Elevation profile" MVP item and TRIP_PLANNING.md for the design history) — bulk-download USGS 3DEP 1-meter DEM tiles for the corridor, confirmed to exist at prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/1m/ (same S3 infrastructure as the topo quads), and sample elevation densely along the actual centerline geometry - not just at the existing 4,395 half-mile markers, which would under-count real gain/loss the same way other hiking apps' sparse sampling does. Mirrors fetch_topo_quads.py + spike_raster_mosaic.py's existing bulk-download-then-process-locally pattern rather than a live point-query API.
  • Corridor computation — generate the 30-mile buffer once (ST_Buffer + ST_Union over the centerline + waypoints). (Done — see pipeline/spike_corridor.py.)
  • Clip & join, raster side proven at real scale (2026-07-25) — see pipeline/spike_raster_mosaic.py: mosaics the real 1,654 downloaded US Topo quads (14GB) per corridor-intersecting grid cell, reprojecting each from its native (varying) UTM zone via a lazy WarpedVRT, then clips to the actual corridor polygon. POI-side join (ATC + OSM/opentrail + NHD into one schema) still open — see Unified POI schema below.
  • Unified POI schema — one schema for water/shelter/campsite/crossing/resupply, designed to not bake in AT-only or NYNJTC-only assumptions (value #7), even though only the AT is in scope for v1.
  • Export — base map as PMTiles, POI layers as GeoJSON/FlatGeobuf. Check package sizes are reasonable for a phone download. Each output artifact gets its own content hash (SHA256) — see Publish below for why (per-artifact, not one hash for everything).
  • Extended-context background beyond the 30-mile corridor, validated 2026-07-27 — give hikers some visual context beyond the core corridor (panning out, or bailing out to a town) without expanding our own USGS raster pipeline. Real-world tested: Protomaps' basemap (an OpenStreetMap-derived vector tileset, same PMTiles format we already use) extracted for a wide box around the whole corridor comes to just 57MB at max zoom 9 (1.6MB at zoom 6, up to 293MB at zoom 11) — a small fraction of the corridor's own 1.18GB raster archive, and self-hostable on the same R2 bucket (no API key, no metered billing). Confirmed it's a genuine, full-featured basemap (9 vector layers: roads, water, places, POIs, buildings, boundaries, landcover, landuse, earth), not a stripped-down stub.
    • Not yet decided: final zoom range to ship (z9/57MB is the tested baseline; higher zooms cost more but work fine at the CLI level too — one attempt hit a transient network blip during testing, not a real blocker).
    • Follow-up requirement, easy to forget: Protomaps' basemap data requires visible "© OpenStreetMap" attribution (ODbL license) somewhere in the app UI once this ships — a small client-side task, not a data or cost concern.
    • Not yet started: wiring this into the client (MapLibre style, layered beneath the corridor raster) — the client doesn't exist yet, see Phase 2.
  • Publish, change-aware plan decided 2026-07-25 — push versioned packages to Cloudflare R2, but only the artifacts that actually changed, and never publish a new version if nothing did. Full plan in TECHNICAL_ARCHITECTURE.md's "Data pipeline" section. Blocked on one decision that needs to happen once, not twice: hiker download chunking granularity (per-state? per section? one corridor-wide package?) — this determines the hashing/versioning granularity too, and is the same open question as "Offline download flow" below. Decide there first, then implement here.
  • Reach out to opentrail.org's maintainer (GitHub: austinwritescode) to confirm data-reuse terms directly. Their /api/getData endpoint (AT/PCT/CDT waypoints incl. resupply-relevant POIs) has no LICENSE file anywhere in the repo, so reuse rights aren't formally confirmed - the maintainer reportedly called it "open data" in a Reddit post (r/Ultralight), which is why we're proceeding with location/POI data only for now (deliberately excluding their user comments - those are personal contributions from named individuals, a separate consent concern from copyright), but this needs a real answer, not an informal comment-thread reference. Good opportunity for reciprocity too - OurHike intends to be open data itself (value #3, #6; see FEATURES.md "Data openness & portability"), so this can be a two-way conversation once there's something functional to show them.
  • Investigate NHD flowline stream-crossings as a water-source candidate list, visually + against verifiable sources. Goal: OSM water tagging tops out around 178-326 near the trail (tested 100m-800m) - nowhere close to FarOut/the AT Guide's 1,100+. Researched whether USGS NHD (hydrography, already in the pipeline for the base map) could close that gap by algorithmically finding stream crossings instead of relying on point tagging. Findings so far (2026-07-25):
    • NHD's "Spring" point layer specifically is sparse near the AT (only 99 within 5mi of centerline) - not a useful source on its own.
    • NHD flowline (stream/river) data crossing the trail is much richer. Buffer/proximity-based counting (nearby streams within some radius) produces numbers 2-5x larger than the 1,100 target depending on radius (2,500-5,900 "crossing events" after declustering) - real, but includes a lot of minor/unnamed streams a guidebook wouldn't bother listing, so it overshoots rather than undershoots.
    • True geometric intersections (where the trail's line literally crosses a stream's line, no buffer/clustering judgment call) is the cleanest cut: 841 total crossings, 568 perennial, 201 intermittent, 72 other. Notably below 1,100, but a solid, unambiguous starting number - no arbitrary radius to defend.
    • Next steps before this becomes a real pipeline source: (1) visually inspect a sample of the 841 crossings on a map to sanity-check they're real/sensible, not an artifact of positional misalignment between ATC's centerline and NHD's flowlines; (2) cross-reference against a verifiable source (e.g. the AT Guide's ~989 water-tagged mile-table entries, kept in personal_reference/ - see that folder's README for why it's not in the pipeline) to see how much real overlap vs. gap there is, rather than just comparing raw counts.
  • Update cadence + change detection, decided 2026-07-25: check weekly, only actually re-fetch what changed. Each fetch script now checks a cheap upstream signal before doing the real (slower/larger) data pull, and skips entirely if nothing changed:
    • fetch_all.py (ATC ArcGIS layers) — checks each layer's editingInfo.dataLastEditDate (a lightweight metadata-only request) against the value recorded in data/raw/manifest.json from the last run; skips the full paginated feature fetch if unchanged. Verified: second consecutive run correctly skipped all 12 sources.
    • fetch_opentrail.py — uses real HTTP conditional requests (If-None-Match / ETag, which the API's own README documents support) rather than reimplementing change detection - a 304 response means skip. Verified working.
    • fetch_topo_quads.py already had this per-quad via S3 Last-Modified headers against its own manifest (built in from the start, no change needed).
    • Still open: where this actually gets triggered weekly (manual run by a maintainer vs. a scheduled job e.g. GitHub Actions cron) - the cadence and "don't do unnecessary work" logic are now real, but nothing calls these scripts on a schedule yet.

Phase 2 — Client app (MVP)

  • Scaffold the PWA — React + TypeScript, Vite, basic service worker + manifest.
  • Map rendering — MapLibre GL JS reading PMTiles directly in-browser (via the pmtiles library's MapLibre protocol handler).
  • Offline download flow, chunking decided 2026-07-28: whole corridor, one package — a hiker downloads the entire trail's data at once, not picked-by-section; store via Cache API/IndexedDB. This is the core "why would I use this instead of a browser tab" feature. Same answer applies to Phase 1's change-aware Publish step (same hashing/versioning granularity question, decided once here, not twice).
  • Let hikers choose a background-map quality/size tradeoff in settings — offer max zoom 11/12/13 as an explicit choice, not just a fixed pipeline default, so hikers on limited data/storage can trade detail for a smaller download. Real measured whole-corridor sizes: z11 ~64MB, z12 ~314MB (current pipeline default, see pipeline/README.md), z13 ~1.18GB — z13 alone is 73% of the full archive's bytes despite only matching the source's native ~11m/pixel resolution, not exceeding it. Mechanism still open (separate pre-built size variants published per zoom vs. the client selectively caching only up to a chosen zoom from one hosted archive, which PMTiles' range-request design may make possible without publishing redundant copies) — needs the client to exist first, not started. Same settings screen also gets a background source picker, design drafted 2026-07-28: see MAP_OPTIONS.md — USGS's own live tile service researched and recommended as the one genuinely free, no-terms-conversation-needed live option.
  • "You are here" — GPS via the browser Geolocation API, shown live on the offline map. Map chrome (legend/scale/locate/zoom) spec'd 2026-07-28: see MAP_OPTIONS.md — MapLibre's built-in GeolocateControl wraps this directly. The compass button reusing that same chrome, and full heading-up auto-rotate, are detailed 2026-07-28: see UX_CUSTOMIZATION.md — auto-rotate needs real custom code (MapLibre has no built-in heading-follow) plus, for the Capacitor iOS shell specifically, a native WKUIDelegate callback or compass permission is silently denied.
  • POI browsing/search — filter/list water sources, shelters, campsites, resupply points, crossings over the GeoJSON layer.
  • Outdoor usability pass — test readability in sunlight glare, one-handed/gloved use. Ties directly to value #4 (trustworthy above all) — a map that's unreadable at a junction fails at its one job. Dark-mode auto-detection (prefers-color-scheme) is a near-free detail of this pass, not a separate build: see UX_CUSTOMIZATION.md — worth noting dark mode doesn't solve the glare problem this pass actually owns; they're different problems despite sounding related.
  • Elevation profile chart, moved into MVP 2026-07-28 — interactive elevation profile + gain/loss + Naismith's Rule time estimate for any selected stretch of trail, consuming the dense elevation data from Phase 1 above. See FEATURES.md's MVP list for the full rationale (competitive parity with FarOut, plus a real safety angle).

Phase 3 — App store packaging

  • Wrap with Capacitor — generate iOS + Android shells from the PWA build (npx cap add ios / android).
  • iOS build & TestFlight — build via Xcode on the Mac; beta test through TestFlight before public submission. Apple Developer account ($99/yr — check nonprofit fee waiver eligibility first).
  • Android build & internal testing — build via Android Studio/Gradle; Google Play Developer account ($25 one-time).
  • Store listing assets — screenshots, description, and a privacy policy (required by both stores, and necessary anyway since the app uses location — keep it plain and honest, per value #4).

Phase 4 — Launch readiness

  • Real-trail field testing — offline mode, GPS accuracy, battery drain, tested on an actual section of trail, ideally by NYNJTC or ATC volunteers.
  • Web-only donation/payment flow — basic Stripe (or similar) checkout on the web version only, nothing equivalent in the wrapped app.
  • Inheritability docs — keep FEATURES.md/TECHNICAL_ARCHITECTURE.md current, add a CONTRIBUTING.md / setup guide aimed at "the next club," not just this one.
  • Soft launch with NYNJTC — first real users, gather feedback, iterate before wider release.

Deferred (Phase 5+, post-MVP — see FEATURES.md)

  • Community condition reporting & maintainer verification
  • Trail magic features
  • Multi-club admin/config tooling
  • Weather integration (design drafted 2026-07-28, see features/HIKER_SAFETY.md)

Immediate next step

Phase 0 is mostly "make some calls / sign up for accounts" — the first coding task, whenever you're ready to pick it back up, is the DuckDB spatial spike at the top of Phase 1.