Skip to content

CHANGELOG.md

Latest commit

 

History

History
187 lines (183 loc) · 15.4 KB

CHANGELOG.md

File metadata and controls

187 lines (183 loc) · 15.4 KB
Raw

Changelog

All notable changes to this project are documented here. The format is based on Keep a Changelog; versioning follows SemVer 2.0 per the portfolio release-and-versioning standard once the first tag lands.

[Unreleased]

Added

  • Expanded ATS adapter library (ADR-0031): 46 more adapters ported from the source repo (48 total, plus a Playwright-based fallback scraper), each individually read and classified before porting — platform clients (Workday, iCIMS, ADP Workforce Now, Oracle Recruiting Cloud, Paradox, and more), generic fallback strategies for boards with no dedicated ATS (sitemap mining, embedded-JSON-state walking, an opt-in Bedrock LLM extractor, poll/render.py's Playwright JS-render scraper), and four approved niche community job boards. 8 adapters needed a specific personal reference removed before porting (an internal doc cross-reference, a named-companies research note, a hardcoded single-employer carve-out, and — the one direct hit — the repo owner's own name in a ranking-preference comment); 2 adapters (calcareers, a California state-jobs board hardcoding the owner's own tracked civil-service classifications, and ucr_json, a single-university board) were excluded rather than redacted. poll/http.py gained post_json/get_text/is_public_http_url/HttpError/BlockedUrlError; two new small shared modules, poll/urls.py and a deliberately-simplified poll/identity.py, back the adapters that need host/employer-identity checks. Posting's 10-field ADR-0025 shape is unchanged — every adapter trims to it rather than growing it. 48 new adapter test files plus direct unit tests for poll/http.py/urls.py/identity.py; coverage 85.7% (gate: 85%). The purge gate caught two more real leaks beyond the manual review (a board-id example matching the owner's own strong-token list; two "verified live <date>: <district> <count>..." sentences naming real institutions) — both fixed, both documented in the ADR as a reason the automated gate isn't optional even after a careful manual pass.
  • First UI: web/ (Astro, static output, ADR-0030) — Cognito Hosted UI sign-in via authorization-code + PKCE (RFC 7636), landing on a page that calls GET /me and shows the result. Auth runs entirely client-side (no framework added for three pages of DOM updates), matching ADR-0022's no-client-secret app client design. generateCodeChallenge verified byte-for-byte against RFC 7636 Appendix B's own worked example. Found and fixed two real bugs: Node ≥22's experimental global localStorage silently shadowing happy-dom's working implementation in Vitest (fixed via NODE_OPTIONS=--no-experimental-webstorage), and a latent bug in tests/test_purge_gate.py's email-domain allowlist (predating this change, from M1.6) that compared the wrong string and would have flagged every @example.com fixture — fixed and verified a real non-allowlisted address still gets caught. docs/THREAT-MODEL.md gains T11 (token theft via XSS — open, not yet mitigated). Purge gate extended to web/src/web/ test the same day this tree was created. 35 new tests (Vitest); CI gained a web job.
  • Real Scorer implementation (scoring/bedrock.py, ADR-0029): BedrockScorer calls a Bedrock Anthropic model via invoke_model with the existing fenced prompt (build_scoring_prompt) and validates every field of the response before constructing a ScoreResult — model output gets the same untrusted-content treatment ADR-0009 already gives input. Coerces safe numeric types, range-checks score/facet values, drops hallucinated facet ids as noise, fails closed (MalformedModelResponseError) on anything else, including a malformed outer API envelope. StubScorer stays exactly what it always was: pipeline plumbing, never real judgment. No live Bedrock call was made (moto doesn't simulate model inference); tests verify the exact prompt sent and response-parsing behavior against a hand-built fake client instead. docs/THREAT-MODEL.md gains T10 (malformed/adversarial model output). boto3-stubs[bedrock-runtime] added as a dev dependency.
  • Alert rendering and SES delivery (notify/render.py, notify/ses_sender.py, ADR-0028): closes the "nothing renders or sends" gap flagged since ADR-0025. Rendering is pure and always runs, even for a verdict that will never send — a future dry-run preview UI has something real to show without SES ever being touched. deliver sends only for verdict == "alert", checking channel enablement, dry-run, and quiet-hours deferral against NotifyDecision — never re-deriving any of them. SESClient typed via boto3-stubs + TYPE_CHECKING, the same fix DynamoStore (ADR-0023) needed for boto3-stubs' PEP 692 Unpack[TypedDict] kwargs. moto[ses]/boto3-stubs[ses] added as dev deps.
  • Scheduled polling bridge (scheduled_poll.py, ADR-0027): connects due-work scheduling (ADR-0018), the watchlist (ADR-0026), and the pipeline (ADR-0025) — three subsystems that previously only worked independently. sync_watchlist_due_work idempotently schedules every watchlist entry; run_due_item resolves one dispatched item back to its WatchlistEntry, runs the pipeline, and reschedules via the org's own cadence tier (resets on a hit, backs off when a cycle finds nothing new). A removed org fails soft — unschedules instead of raising. End-to-end tested: watchlist entry → scheduled → fair-share-dispatched → pipeline runs → cadence adapts across two real cycles against the live-recorded GitLab fixture.
  • Per-user org watchlist (tenancy/watchlist.py, ADR-0026): CRUD over the orgs entity, closing the "not-yet-built" gap ADR-0025 left. WatchlistEntry moved from pipeline.py to tenancy/watchlist.py in the process — a layering fix (orchestration depends on the primitive, never the reverse) rather than a new decision. No entitlement check inside add; the caller composes EntitlementsService.assert_org_capacity, the same decoupling every cross-service seam in this codebase keeps.
  • First real vertical slice: poll → filter → score → notify, end to end (poll/, pipeline.py, tenancy/seen.py, ADR-0025). Two ATS adapters ported from the source repo (Greenhouse, Lever — both public, unauthenticated JSON APIs, verified personal-data-free before porting) feeding pipeline.run_for_org, which wires every subsystem this milestone built (policy filters, Scorer, notify.decide, tenancy) into one real pipeline. Test fixtures are live-recorded from GitLab's public Greenhouse board and Lever's own public demo board, not fabricated; testing against them surfaced and fixed a real double-HTML-encoding bug in Greenhouse's content field that the source repo's own strip_html doesn't fully handle either. The purge gate caught one real leak during development (a User-Agent string carrying the owner's GitHub username) before it landed. Verified against live GitLab data beyond the deterministic fixture suite (204 real postings, 166 scored). docs/THREAT-MODEL.md gains T9 (SSRF via a malicious board id — mitigated by charset validation + https-only enforcement).
  • First live request path (infra/lib/api-stack.ts, lambda_handlers/me.py, ADR-0024): GET /me behind an HTTP API with a Cognito HttpUserPoolAuthorizer — the JWT is verified before the Lambda ever runs, and the handler trusts only the verified claims. Idempotently provisions the caller's workspace and returns plan/status, exercising ProvisioningService/EntitlementsService/DynamoStore together end to end; no new business logic, only wiring already-tested services. Closes T1's open half in docs/THREAT-MODEL.md for this route. No Docker bundling: the handler stays dependency-light (boto3 only, ships pre-installed in Lambda), and Code.fromAsset zips src/ verbatim. IAM is table-scoped, not yet entity-scoped — tracked open, not silently assumed complete.
  • DynamoDB Store adapter (tenancy/dynamo_store.py, ADR-0023): implements the same protocol InMemoryStore does against a real (or moto-mocked) DynamoDB table keyed exactly like entities.scoped_key; transparent float ⇄ Decimal conversion at the boundary; list's prefix scan is a paginated Query with begins_with, never a Scan. The tenant-isolation adversarial suite reruns against this backend in its own additive file (tests/test_dynamo_store_isolation.py), never touching the existing merge-gated file. Closes the gap store.py's docstring named since ADR-0016.
  • Control-plane Cognito identity boundary (infra/, ADR-0022, extends ADR-0021/ADR-0003): ControlPlaneStack — self-service sign-up with required email verification, a 12-character password floor, MFA optional (TOTP only, SMS explicitly disabled), and a public SPA app client (no secret, authorization-code/PKCE flow, preventUserExistenceErrors on). Hosted UI callback/logout URLs are never defaulted outside dev; stage/prod synth fails closed without them explicitly set. EnvName factored into a shared lib/env.ts for both stacks. Infra Jest suite: 23 tests total (up from 8).
  • First infrastructure-as-code (infra/, CDK v2 TypeScript, ADR-0021): a DataPlaneStack making the Store/DueWorkIndex Python protocols deployable — one DynamoDB table for every TenantRepository entity (key schema mirrors entities.scoped_key exactly), and a separate table for DueWorkIndex with a date-bucketed DueIndex GSI for the dispatcher's cross-tenant due-before scan. No AWS account ID is ever a literal in source (roadmap item A8's exact failure mode); envName (dev/stage/prod) is the one validated, fail-closed parameter, controlling RemovalPolicy (retain in prod, destroy elsewhere). Jest + aws-cdk-lib/assertions unit tests against the synthesized template; CI gained an infra job (npm ci && build && test && cdk synth, no AWS credentials needed), fulfilling M0's original "cdk synth on PRs" exit criterion. The purge gate (tests/test_purge_gate.py) now scans infra/{bin,lib,test} alongside the Python engine, from day one rather than as a follow-up.
  • M4 tenant-fairness soak test (roadmap M4 exit criterion, first half): 200 synthetic tenants with heterogeneous plans/thresholds/quiet-hours driven through the due-work index, fair-share dispatcher, source-yield ledger, and notify decision together; asserts the dispatched (tenant, org) pairs exactly reproduce what was scheduled (no cross-talk) and that identical posting scores diverge across tenants' own thresholds. The exit criterion's other half — p95 poll latency within 2x the single-user baseline — needs real polling infrastructure this repo doesn't have yet and stays open.
  • Per-user source-yield tracking (roadmap M4, ADR-0020): capped 72-outcome rolling ledger per (user, org) recording posting_seen/applied/interview/offer/rejected/ignored events; a hot/warm/cold tier recommendation that stays at the neutral default below a minimum sample size, so a source isn't judged on too little history. Decoupled from scheduling.cadence by a plain string contract, not an import.
  • Tenant-aware notify decision (roadmap M4, ADR-0019): pure alert/digest/suppressed routing over per-user thresholds, with dry-run and channel eligibility carried as orthogonal axes so a downgraded Free user's high-scoring posting is still alert, is_dry_run=True — never a silent fourth suppression tier. UTC-native quiet hours defer (never drop) alerts.
  • Due-work scheduling and fair-share dispatch (roadmap M4 foundation, ADR-0018, extends ADR-0005): yield-adaptive cadence policy (hot/warm/cold/digest_weekly presets) replacing poll_tiering.yaml's global constants; a cross-tenant DueWorkIndex — the "operator GSI" the roadmap calls out, with tenant-validated writes and a structurally content-free row shape; a round-robin fair-share Dispatcher draining free/plus/pro lanes with per-user noisy-neighbor caps and priority-ordered cross-lane shedding under a global batch limit; adversarial fairness suite (unit + Hypothesis property tests) as a merge gate alongside the tenant-isolation suite.
  • Control plane (roadmap M3, ADR-0017): plan catalog with structural Free-tier caps; entitlements service with fail-closed capability checks, org-capacity limits, capped transition history, and the downgrade-to-dry-run invariant; Stripe-shaped billing webhook handler with signature-verifier boundary and per-event-id idempotency + dunning ladder; usage metering accumulators; idempotent workspace provisioning.
  • Tenancy primitives (roadmap M2.1–M2.3, ADR-0016): validated TenantContext, tenant-safe store protocol (no raw-key API) with in-memory implementation, deny-by-default repository with body-only reads and closed entity catalog; model-based adversarial cross-tenant fuzz suite as a merge gate.
  • Per-user services on the tenancy primitives: settings service with layered resolution + optimistic versioning (M2.4); budget ledger with pre-invocation daily/monthly caps under a platform ceiling (M2.6); credentials-vault skeleton behind a Cipher boundary with masked views and rotation bookkeeping (M2.5).
  • Initial threat model (docs/THREAT-MODEL.md) with STRIDE dispositions and open items (M2.7).
  • Scoring subsystem under a fixed contract, stubbed-engine-first (ADR-0015, roadmap M1.4): runtime-checkable Scorer protocol and ScoreResult; rubric-as-config with six built-in archetype rubrics (schema-v2, weight/band semantics validated); fully-fenced prompt assembly with untrusted-content sanitization; deterministic StubScorer; conservative shadow-replay activation gate over golden sets.
  • Purge gate + stranger-parity harness (roadmap M1.6): structural no-personal-data checks over engine source, optional strong-token enforcement via untracked .purge-tokens.local, and ≥3 synthetic stranger profiles run end-to-end through the policy stack with divergence assertions.
  • Values-screening preset packs (openjobradar.policy.screen_organization): neutral none default, data-defined public_interest pack, user-extensible denylist with additive composition and evidence-cited denials (ADR-0014, roadmap M1.5).
  • Declarative hard-filter DSL (openjobradar.policy.evaluate_hard_filters): clearance stance, comp floor, excluded seniority/archetypes, and current-employer suppression as pure functions over user config with collected, explainable findings (ADR-0013, roadmap M1.3).
  • Declarative location-policy engine (openjobradar.policy): pure evaluation of posting locations against user-owned location_policy config with explainable verdict rules (ADR-0012, roadmap M1.2).
  • Product roadmap for the multi-tenant SaaS conversion (docs/OPENJOBRADAR-ROADMAP.md, Path B).
  • Architecture decision records 0001–0010 (license, pooled tenancy, userId identity, deny-by-default storage, due-work scheduling, per-user config documents, KMS credential vault, staged email-sender strategy, untrusted prompt content, environment promotion).
  • Schema-validated per-user configuration core: profile/settings JSON Schemas (v2), fail-closed validation with aggregated errors, prototype v1→v2 profile migration, layered settings resolution (platform < plan < user < session override) with source tracking.
  • CI gates: ruff, mypy, pytest with branch-coverage floor; make verify parity.
  • Portfolio standards registration and in-repo conformance declarations.