Skip to content

ROADMAP.md

Latest commit

 

History

History
106 lines (83 loc) · 26.6 KB

ROADMAP.md

File metadata and controls

106 lines (83 loc) · 26.6 KB
Raw

OurHike — Roadmap / Build Plan (Draft v1)

Companion to OurHikeValues.md, FEATURES.md, and TECHNICAL_ARCHITECTURE.md. Ordered, phased list of what needs to get built and how, from where we are now (decided architecture) to a shippable v1. Expect this to shift as decisions firm up — check items off as they're done, add detail as it emerges.

Feature design docs

Fourteen features have full design drafts, plus one consolidated reference doc, written 2026-07-28 (Feature Gating added 2026-07-31) and now organized in features/ so nothing gets lost. Each is already linked in context from FEATURES.md and from the phase items above — this section is just the one place that lists them all together. A cross-feature alignment review on 2026-07-28 moved Authentication, Report a Problem, Map Options' closures, and Hiker Safety's warnings/wrong-way alert into v1 MVP (see TECHNICAL_ARCHITECTURE.md's revised Backend section) — the notes below reflect that revision, not the original scope each doc launched with:

  • features/AUTHENTICATION.md — Google/Apple/email sign-in, verification, optional MFA. Moved into v1 MVP 2026-07-28 (browsing the map still needs no account - this exists so a moderator/reporter can be identified) - still foundational for Segments, Volunteering, and Report a Problem too.
  • features/FEATURE_GATING.md — chapter-aware feature flags and experiments with safe stable defaults, offline-friendly evaluation, and transparency for users behind a gate. A cross-cutting platform capability to make all later features safer and evidence-backed.
  • features/TRAIL_BLAZE_COLORS.md — render the trail line in its real painted blaze color, with a neutral fallback when unknown. A v1 MVP correctness detail, not Post-MVP.
  • features/SEGMENTS.md — hierarchical Hike → Segment tree so hikers can plan and track a thru-hike, section, or day-hike however suits them. Post-MVP (Extras).
  • features/TRIP_PLANNING.md — builds on Segments: day-hiker waypoint planning, bulk multi-day date shifts, POI-aware planning assistance. Post-MVP (Extras).
  • features/REPORT_A_PROBLEM.md — hiker-submitted condition reports (blow downs, trash, bad hikers, flooding, shelter repair, animals), with "bad hikers" routed internal-only. Moved into v1 MVP 2026-07-28 alongside Map Options' closures and Hiker Safety's warnings, both of which reuse this exact moderation mechanism.
  • features/VOLUNTEERING.md — club work-project management, plus upcoming projects shown on the map to encourage hikers to join one. Post-MVP (Trail magic).
  • features/MAP_OPTIONS.md — user-controlled background tile source, roads/sidewalk-walkability overlay, DuckDB-backed snap-to-trail for Segments, and trail-closure marking/display. Trail closures moved into v1 MVP 2026-07-28, and its map-chrome section (legend/scale/locate/zoom) is a v1 MVP detail spec too; background tile options, roads/sidewalks, and snap-to-segment stay Post-MVP.
  • features/HIKER_SAFETY.md — moderator-escalated serious warning pins, a configurable anonymity window for comments, a responsibly-sourced NWS weather-alert relay and elevation-aware daily conditions, and a deliberately conservative wrong-way/off-trail alert scoped as the only notification OurHike ever sends. Warning pins and the wrong-way alert moved into v1 MVP 2026-07-28; the anonymity window and weather integration stay Post-MVP.
  • features/DATA_NUDGES.md — low-friction, non-gamified prompts to keep water/shelter/resupply data fresh: no notification of any kind, just visual map prominence for stale POIs and open reports, self-limiting the moment anyone contributes. Post-MVP (Community reporting).
  • features/UX_CUSTOMIZATION.md — light/dark mode, metric units, which waypoints display and at what detail, and auto-rotate/compass. Splits persistent settings from on-map controls up front; auto-rotate is real Post-MVP work given the platform constraints, the rest is mostly MVP-detail or light Post-MVP settings polish.
  • features/ONBOARDING.md — a deliberately minimal first-run flow (value-prop, the download choice, contextual location permission) scoped as v1 MVP; trail names, a settings mention, and helpful-info tips are a second, Post-MVP tier that waits on Authentication and UX Customization existing first.
  • features/COMMUNITY_BUILDING.md — forming a "Tramily" and sharing routes, on-demand/periodic safety check-ins, and content-attached "@" mentions. This project's biggest privacy-vs-connection tension yet, resolved as a deliberate, scoped exception (opt-in/mutual/revocable/minimal-retention) rather than loosening the stance Hiker Safety and Data Nudges already took. Post-MVP.
  • features/IDENTITY_AND_PRIVACY.md — not a feature, a consolidated reference written during the 2026-07-28 alignment review: ties together identity/privacy design scattered across five docs (who sees what, on which surface, governed by which mechanism) and replaces five separate small settings models with one canonical UserPreferences.
  • features/PRICING_MODEL.md — extends the existing Business Model section in FEATURES.md: a thru-hike pass (validated against FarOut's real $74.99 AT guide price), a regional pass tied to Multi-club support becoming a real feature, a volunteer exemption blocked on Volunteering's not-yet-built hour-tracking, and an annual all-access ceiling. Reuses the existing MVP/Post-MVP line as the paywall boundary rather than inventing a new one. Post-MVP, timing deliberately undecided.
  • features/FEATURE_GATING.md — added 2026-07-31. Per-chapter feature flags and A/B experimentation, recommending self-hosted GrowthBook specifically for its "app should never depend on the flag tool being available" design philosophy — a hard requirement here, not a nice-to-have, given hikers rely on this app with no signal. Client-side evaluation is always local/offline against a last-synced cache, with a single universal fail-open default (current production behavior) for every failure mode. Post-MVP, but recommended as the first thing built after MVP launch stabilizes, since every feature built after it benefits from real evidence instead of a guess. Overlaps with and partially satisfies the Deferred "Multi-club admin/config tooling" item below.

Phase 1 — Data pipeline

  • DuckDB spatial spike. Prove the core operation works before building on top of it: buffer the AT centerline by 30 miles, clip real data against it, export the result. (Done — see pipeline/spike_corridor.py. Buffered the full 3,025-segment centerline by 30mi, unioned into one valid corridor polygon (~81,138 sq mi), clipped real ATC campsite/shelter data against it — all 232 campsites and 280 shelters fell within the corridor, as expected for official on-trail sites. Used ATC data only, not OSM — see note below on rethinking OSM's role first. Hit and documented an ST_Transform axis-order gotcha in pipeline/README.md, worth knowing before anyone else touches that call.)

  • Think through open-source trail data strategy more deliberately before building more ingestion. Prompted by realizing the OSM pull was about to become "grab roads because the architecture doc says so" rather than solving a real hiker need — and by USGS's National Digital Trails page on the many ways to access their trails dataset. Worth deciding deliberately: what background/reference map layer to use (raster vs. vector, and from where — USGS, OSM, something else), and which specific gaps in ATC's own data (confirmed gap: no water-source or general resupply layer) get filled by which external source. Deserves real thought, not an MVP-speed decision.

  • Ingestion scripts — pull raw extracts from ATC (POIs), USGS (elevation/topo), OSM (roads/land cover) into formats DuckDB reads directly (GeoJSON/Shapefile/GeoPackage/FlatGeobuf). (ATC side started — see pipeline/: discover_sources.py finds layer URLs from ATC's public ArcGIS map, fetch_all.py pulls all 9 registered layers. USGS/OSM ingestion not started.)

  • Elevation data, moved into MVP 2026-07-28 (see FEATURES.md's "Elevation profile" MVP item and TRIP_PLANNING.md for the design history) — bulk-download USGS 3DEP 1-meter DEM tiles for the corridor, confirmed to exist at prd-tnm.s3.amazonaws.com/StagedProducts/Elevation/1m/ (same S3 infrastructure as the topo quads), and sample elevation densely along the actual centerline geometry - not just at the existing 4,395 half-mile markers, which would under-count real gain/loss the same way other hiking apps' sparse sampling does. Mirrors fetch_topo_quads.py + spike_raster_mosaic.py's existing bulk-download-then-process-locally pattern rather than a live point-query API.

  • Corridor computation — generate the 30-mile buffer once (ST_Buffer + ST_Union over the centerline + waypoints). (Done — see pipeline/spike_corridor.py.)

  • Clip & join, raster side proven at real scale (2026-07-25) — see pipeline/spike_raster_mosaic.py: mosaics the real 1,654 downloaded US Topo quads (14GB) per corridor-intersecting grid cell, reprojecting each from its native (varying) UTM zone via a lazy WarpedVRT, then clips to the actual corridor polygon. POI-side join (ATC + OSM/opentrail + NHD into one schema) still open — see Unified POI schema below.

  • Unified POI schema — one schema for water/shelter/campsite/crossing/resupply, designed to not bake in AT-only or NYNJTC-only assumptions (value #7), even though only the AT is in scope for v1. Design decided 2026-07-29, not yet built: the planned dim_pois dbt mart (see below) unifies ATC shelters/campsites + opentrail waypoints as its first slice — not yet all ~12 ATC sources.

  • dbt-based ELT transform layer, design decided 2026-07-29, not yet built. Resequenced 2026-07-31: the first major post-launch initiative, not MVP-critical-path work. It was mistakenly pulled toward the MVP launch effort once already — it isn't in LAUNCH_CHECKLIST.md and doesn't need to be; browsing/downloading/publishing the map works without it. It's listed here under Phase 1 for design continuity (it is Phase 1's data-transform layer), but the real trigger for building it is timing-driven, not pipeline-sequencing-driven: NYNJTC's own non-AT trail network is expected to be onboarded on a real, near-term timeline, and dbt Phase D's registry pattern (below) is what lets that onboarding be "new rows + new staging models," not a second parallel ad hoc pipeline. Start Phase A/B right after MVP ships (Phase 3/4 below), and finish before NYNJTC's own trail data work begins — doing it after would mean either duplicating today's one-off-script approach for a second trail or retrofitting dbt under time pressure while a club is already expecting service.

    Raw ATC/opentrail data loads into DuckDB (planned load_raw.py) before transformation (ELT, not just ETL); dbt (pipeline/dbt/, not yet scaffolded) transforms it via staging -> intermediate -> marts SQL (dbt_utils, dbt_project_evaluator, dbt-codegen), tested and documented (dbt yml descriptions sourced from sources.json/fetch_opentrail.py's ICON_LEGEND, dbt docs generate). SQLFluff lints the SQL; both dbt tests and lint are planned to run in CI. Raster pixel data is deliberately excluded from the warehouse - only lightweight metadata loads. See pipeline/DBT.md for the full design and phased rollout plan (Phase A: scaffold + first vertical slice; Phase B: reconcile against real fetched data, expand to the remaining ATC sources; Phase C: CI required-check promotion; Phase D: pattern for onboarding a second trail — the phase this resequencing is really about).

  • Export — base map as PMTiles, POI layers as GeoJSON/FlatGeobuf, including per-feature blaze_color normalization (see TRAIL_BLAZE_COLORS.md — decode each source's real coded domain during export, flat per-source default for uniform sources like centerline, neutral fallback + warning for anything that doesn't decode). Check package sizes are reasonable for a phone download. Each output artifact gets its own content hash (SHA256) — see Publish below for why (per-artifact, not one hash for everything).

  • Extended-context background beyond the 30-mile corridor, validated 2026-07-27 — give hikers some visual context beyond the core corridor (panning out, or bailing out to a town) without expanding our own USGS raster pipeline. Real-world tested: Protomaps' basemap (an OpenStreetMap-derived vector tileset, same PMTiles format we already use) extracted for a wide box around the whole corridor comes to just 57MB at max zoom 9 (1.6MB at zoom 6, up to 293MB at zoom 11) — a small fraction of the corridor's own 1.18GB raster archive, and self-hostable on the same R2 bucket (no API key, no metered billing). Confirmed it's a genuine, full-featured basemap (9 vector layers: roads, water, places, POIs, buildings, boundaries, landcover, landuse, earth), not a stripped-down stub.

    • Not yet decided: final zoom range to ship (z9/57MB is the tested baseline; higher zooms cost more but work fine at the CLI level too — one attempt hit a transient network blip during testing, not a real blocker).
    • Follow-up requirement, easy to forget: Protomaps' basemap data requires visible "© OpenStreetMap" attribution (ODbL license) somewhere in the app UI once this ships — a small client-side task, not a data or cost concern.
    • Not yet started: wiring this into the client (MapLibre style, layered beneath the corridor raster) — the client doesn't exist yet, see Phase 2.

  • Publish, change-aware plan decided 2026-07-25 — push versioned packages to Cloudflare R2, but only the artifacts that actually changed, and never publish a new version if nothing did. Full plan in TECHNICAL_ARCHITECTURE.md's "Data pipeline" section. Blocked on one decision that needs to happen once, not twice: hiker download chunking granularity (per-state? per section? one corridor-wide package?) — this determines the hashing/versioning granularity too, and is the same open question as "Offline download flow" below. Decide there first, then implement here.

  • Reach out to opentrail.org's maintainer (GitHub: austinwritescode) to confirm data-reuse terms directly. Their /api/getData endpoint (AT/PCT/CDT waypoints incl. resupply-relevant POIs) has no LICENSE file anywhere in the repo, so reuse rights aren't formally confirmed - the maintainer reportedly called it "open data" in a Reddit post (r/Ultralight), which is why we're proceeding with location/POI data only for now (deliberately excluding their user comments - those are personal contributions from named individuals, a separate consent concern from copyright), but this needs a real answer, not an informal comment-thread reference. Good opportunity for reciprocity too - OurHike intends to be open data itself (value #3, #6; see FEATURES.md "Data openness & portability"), so this can be a two-way conversation once there's something functional to show them.

  • Investigate NHD flowline stream-crossings as a water-source candidate list, visually + against verifiable sources. Goal: OSM water tagging tops out around 178-326 near the trail (tested 100m-800m) - nowhere close to FarOut/the AT Guide's 1,100+. Researched whether USGS NHD (hydrography, already in the pipeline for the base map) could close that gap by algorithmically finding stream crossings instead of relying on point tagging. Findings so far (2026-07-25):

    • NHD's "Spring" point layer specifically is sparse near the AT (only 99 within 5mi of centerline) - not a useful source on its own.
    • NHD flowline (stream/river) data crossing the trail is much richer. Buffer/proximity-based counting (nearby streams within some radius) produces numbers 2-5x larger than the 1,100 target depending on radius (2,500-5,900 "crossing events" after declustering) - real, but includes a lot of minor/unnamed streams a guidebook wouldn't bother listing, so it overshoots rather than undershoots.
    • True geometric intersections (where the trail's line literally crosses a stream's line, no buffer/clustering judgment call) is the cleanest cut: 841 total crossings, 568 perennial, 201 intermittent, 72 other. Notably below 1,100, but a solid, unambiguous starting number - no arbitrary radius to defend.
    • Next steps before this becomes a real pipeline source: (1) visually inspect a sample of the 841 crossings on a map to sanity-check they're real/sensible, not an artifact of positional misalignment between ATC's centerline and NHD's flowlines; (2) cross-reference against a verifiable source (e.g. the AT Guide's ~989 water-tagged mile-table entries, kept in personal_reference/ - see that folder's README for why it's not in the pipeline) to see how much real overlap vs. gap there is, rather than just comparing raw counts.

  • Update cadence + change detection, decided 2026-07-25: check weekly, only actually re-fetch what changed. Each fetch script now checks a cheap upstream signal before doing the real (slower/larger) data pull, and skips entirely if nothing changed:

    • fetch_all.py (ATC ArcGIS layers) — checks each layer's editingInfo.dataLastEditDate (a lightweight metadata-only request) against the value recorded in data/raw/manifest.json from the last run; skips the full paginated feature fetch if unchanged. Verified: second consecutive run correctly skipped all 12 sources.
    • fetch_opentrail.py — uses real HTTP conditional requests (If-None-Match / ETag, which the API's own README documents support) rather than reimplementing change detection - a 304 response means skip. Verified working.
    • fetch_topo_quads.py already had this per-quad via S3 Last-Modified headers against its own manifest (built in from the start, no change needed).
    • Still open: where this actually gets triggered weekly (manual run by a maintainer vs. a scheduled job e.g. GitHub Actions cron) - the cadence and "don't do unnecessary work" logic are now real, but nothing calls these scripts on a schedule yet.

Phase 2 — Client app & backend (MVP, backend added 2026-07-28)

v1 MVP screens now wireframed (2026-07-28) — see WIREFRAMES.md for the full screen-by-screen spec (map, legend, blaze rendering, downloads, onboarding, reporting, closures, warnings, wrong-way alert, settings, staleness), reconciled against this roadmap and the feature docs. Two spots need attention before/while building: the wireframe's per-section download screens are superseded by this phase's whole-corridor decision below (already resolved, just build to this doc, not the wireframe there); the reporting screens' "unsafe behaviour"/"say thanks" split isn't reflected in REPORT_A_PROBLEM.md's data model yet (still open — see WIREFRAMES.md's Known Deviations #2).

  • Scaffold the PWA — React + TypeScript, Vite, basic service worker + manifest.
  • Map rendering — MapLibre GL JS reading PMTiles directly in-browser (via the pmtiles library's MapLibre protocol handler).
  • Feature gating & experimentation support — chapter-aware feature gates with a safe control fallback, offline-friendly cached feature manifest, and explicit in-app messaging for preview/experiment users. This makes future feature rollout safer and measurable.
  • Offline download flow, chunking decided 2026-07-28: whole corridor, one package — a hiker downloads the entire trail's data at once, not picked-by-section; store via Cache API/IndexedDB. This is the core "why would I use this instead of a browser tab" feature. Same answer applies to Phase 1's change-aware Publish step (same hashing/versioning granularity question, decided once here, not twice).
  • Let hikers choose a background-map quality/size tradeoff in settings — offer max zoom 11/12/13 as an explicit choice, not just a fixed pipeline default, so hikers on limited data/storage can trade detail for a smaller download. Real measured whole-corridor sizes: z11 ~64MB, z12 ~314MB (current pipeline default, see pipeline/README.md), z13 ~1.18GB — z13 alone is 73% of the full archive's bytes despite only matching the source's native ~11m/pixel resolution, not exceeding it. Mechanism still open (separate pre-built size variants published per zoom vs. the client selectively caching only up to a chosen zoom from one hosted archive, which PMTiles' range-request design may make possible without publishing redundant copies) — needs the client to exist first, not started. Same settings screen also gets a background source picker, design drafted 2026-07-28: see MAP_OPTIONS.md — USGS's own live tile service researched and recommended as the one genuinely free, no-terms-conversation-needed live option. This zoom/size choice is also the first real onboarding moment: see ONBOARDING.md — nowhere else for a hiker to make it before the offline map is usable at all.
  • "You are here" — GPS via the browser Geolocation API, shown live on the offline map. Map chrome (legend/scale/locate/zoom) spec'd 2026-07-28: see MAP_OPTIONS.md — MapLibre's built-in GeolocateControl wraps this directly. The compass button reusing that same chrome, and full heading-up auto-rotate, are detailed 2026-07-28: see UX_CUSTOMIZATION.md — auto-rotate needs real custom code (MapLibre has no built-in heading-follow) plus, for the Capacitor iOS shell specifically, a native WKUIDelegate callback or compass permission is silently denied.
  • POI browsing/search — filter/list water sources, shelters, campsites, resupply points, crossings over the GeoJSON layer.
  • Outdoor usability pass — test readability in sunlight glare, one-handed/gloved use. Ties directly to value #4 (trustworthy above all) — a map that's unreadable at a junction fails at its one job. Dark-mode auto-detection (prefers-color-scheme) is a near-free detail of this pass, not a separate build: see UX_CUSTOMIZATION.md — worth noting dark mode doesn't solve the glare problem this pass actually owns; they're different problems despite sounding related.
  • Elevation profile chart, moved into MVP 2026-07-28 — interactive elevation profile + gain/loss + Naismith's Rule time estimate for any selected stretch of trail, consuming the dense elevation data from Phase 1 above. See FEATURES.md's MVP list for the full rationale (competitive parity with FarOut, plus a real safety angle).

Backend & safety features, moved into MVP 2026-07-28

Browsing everything above stays account-free. These five items are what actually need a live backend and real accounts, and only these — see TECHNICAL_ARCHITECTURE.md's Backend section and FEATURES.md's "Safety & community contributions" group for the full reasoning.

  • Backend scaffold — FastAPI + Postgres, per TECHNICAL_ARCHITECTURE.md's revised Backend section. Needed so closures and serious warnings can be verified/moderated by someone — not because the core map needs a server.
  • Authentication — Google/Apple/email sign-in, email verification, optional MFA: see AUTHENTICATION.md.
  • Community condition reports & moderation queue — all six report types (blow downs, trash, bad hikers, flooding, shelter repair, animals): see REPORT_A_PROBLEM.md.
  • Trail closures — marking/display for storm damage and reroutes: see MAP_OPTIONS.md's "Reroutes / closures" section.
  • Serious warning pins & wrong-way/off-trail alert — moderator-escalated warnings, plus a deliberately conservative off-trail alert (still the only push notification OurHike sends): see HIKER_SAFETY.md.

Phase 3 — App store packaging

  • Wrap with Capacitor — generate iOS + Android shells from the PWA build (npx cap add ios / android).
  • iOS build & TestFlight — build via Xcode on the Mac; beta test through TestFlight before public submission. Apple Developer account ($99/yr — check nonprofit fee waiver eligibility first).
  • Android build & internal testing — build via Android Studio/Gradle; Google Play Developer account ($25 one-time).
  • Store listing assets — screenshots, description, and a privacy policy (required by both stores, and necessary anyway since the app uses location — keep it plain and honest, per value #4).

Phase 4 — Launch readiness

  • Real-trail field testing — offline mode, GPS accuracy, battery drain, tested on an actual section of trail, ideally by NYNJTC or ATC volunteers.
  • Web-only donation/payment flow — basic Stripe (or similar) checkout on the web version only, nothing equivalent in the wrapped app. The actual tier/entitlement structure this checkout sells, design drafted 2026-07-28: see PRICING_MODEL.md — thru-hike pass, regional pass, all-access ceiling, and a club-admin-granted volunteer exemption.
  • Inheritability docs — keep FEATURES.md/TECHNICAL_ARCHITECTURE.md current, add a CONTRIBUTING.md / setup guide aimed at "the next club," not just this one.
  • Soft launch with NYNJTC — first real users, gather feedback, iterate before wider release. Distinct from onboarding NYNJTC's own non-AT trail network (a separate, later effort — this item is NYNJTC using the AT app as an early adopter/tester, same data as everyone else). See Phase 1's dbt bullet above, resequenced 2026-07-31: dbt Phase A/B is meant to land in the gap between here and that separate non-AT onboarding, not during it.

Deferred (Phase 5+, post-MVP — see FEATURES.md)

  • Trail magic features
  • Multi-club admin/config tooling
  • Weather integration (design drafted 2026-07-28, see features/HIKER_SAFETY.md)

Immediate next step

Phase 0 is mostly "make some calls / sign up for accounts" — the first coding task, whenever you're ready to pick it back up, is the DuckDB spatial spike at the top of Phase 1.