Skip to content

deployment.md

Latest commit

 

History

History
469 lines (391 loc) · 23.2 KB

deployment.md

File metadata and controls

469 lines (391 loc) · 23.2 KB
Raw

Deployment

Canonical domain, DNS, redirect, analytics, and search migration procedures are defined in domain-portfolio.md.

The API and Base indexer worker must share the generated DISCOVERY_WEBHOOK_SIGNING_KEY through the agent-bounties-discovery Render environment group. The MCP service must not receive it. The API uses it only to derive per-subscription secrets; the worker uses it to sign bounded webhook deliveries. See discovery-subscriptions.md.

The hosted topology is API, MCP, Postgres, and one autonomous Base event-indexer worker. The Base contracts are deployed separately and configured only after review and verification.

Current State

site/protocol.json and deployments/base-mainnet.json are authoritative. They report the capped autonomous-v1 deployment as active and pin its exact factory, implementation, transaction, block, and runtime hashes. Any future address or hash change requires a new deployment record and chain attestation.

The retired operator-signed escrow is recorded only in deployments/base-mainnet-legacy.json. Do not configure it in API, MCP, worker, website, or wallet flows.

Render Blueprint

The root render.yaml creates:

  • agent-bounties-postgres,
  • agent-bounties-api,
  • agent-bounties-mcp,
  • agent-bounties-base-indexer, and
  • agent-bounties-open-competition-v1-indexer.

The two workers are deliberately additive. The original worker continues from the autonomous-v1 factory and cursor. The Open Competition V1 worker starts at its own factory deployment block and writes only versioned open-competition records; it does not migrate or rewrite historical bounty rows.

CLOUD_AGENT_API_KEY is a direct sync: false secret on agent-bounties-api; Render ignores sync: false inside environment groups. MCP proxies the API and must not receive this secret. The exact-SHA controller reconciles nonsecret model settings and copies the optional repository Actions secret CLOUD_AGENT_API_KEY into the API service without logging it. Verify GET /v1/cloud-agent/readiness; see cloud-agent-operations.md.

MoonPay signed external checkout uses two direct sync: false values on agent-bounties-mcp: MOONPAY_PUBLISHABLE_KEY and MOONPAY_SECRET_KEY. Store both as repository Actions secrets. The exact-SHA controller installs them all-or-none, verifies that their test/live prefixes match the repository MOONPAY_ENVIRONMENT variable (default sandbox), and records only redacted configuration evidence. The MCP service returns a signed external MoonPay URL; it does not embed checkout, complete a purchase, transfer crypto, or prove canonical bounty funding.

Farcaster mention ingestion uses five direct sync: false API-service values: NEYNAR_API_KEY, NEYNAR_WEBHOOK_SECRET, NEYNAR_SIGNER_UUID, NEYNAR_BOT_FID, and NEYNAR_BOT_USERNAME. They are all-or-none at runtime. The exact-SHA controller receives two repository secrets and two repository variables, creates or updates the FID-filtered Neynar webhook, reads back its generated signing secret, installs all five API values, redacts every value from evidence, and then requires GET /v1/social/mention-ingestion/readiness to report the database, canonical gate, webhook verification, and reply signer as ready.

Validate before synchronizing:

python scripts\check-render-blueprint.py

Application deployment authority lives in the Render Deploy Recovery GitHub Actions workflow. Native Render auto-deploy is off because the Git provider event stream failed to deliver reviewed main commits reliably. After CI succeeds on a push to main, the workflow:

  1. checks out the exact successful-CI SHA;
  2. verifies it is the latest successful CI revision reachable from main;
  3. resolves all four Render services by exact name and verifies repository, branch, and service type;
  4. if and only if the additive Open Competition V1 indexer is absent, validates exactly one Blueprint bound to this repository, main, and render.yaml, cycles its supported Auto Sync setting, and waits for the exact typed worker; if an otherwise healthy Blueprint remains in sync without materializing that worker, provisions only that allowlisted worker through Render's service API by copying the validated legacy worker's workspace, project environment, and database binding, attaches the exact existing environment group, verifies every nonsecret value by readback, and then revalidates all four bindings;
  5. verifies that the exact shared Base environment group is linked to all four services, derives the Open Competition manifest, verifier catalog, and disabled activation gates from the checked-in mainnet release evidence, and reconciles each nonsecret value through Render with exact readback;
  6. disables any drifted native auto-deploy setting;
  7. reconciles PUBLIC_BASE_URL, MCP_BASE_URL, and WEBSITE_BASE_URL on both public services;
  8. reconciles all nonsecret cloud-agent settings on API and copies the optional GitHub-held model key without including its value in evidence;
  9. reconciles the optional all-or-none MoonPay key pair and environment on MCP without including either key in evidence;
  10. creates or updates the optional FID-filtered Neynar provider webhook and reconciles its bot identity, reply signer, and generated signing secret on API without including their values in evidence;
  11. calls Render's deploy API with the exact commit for API, MCP, and both workers;
  12. waits for all four deploys to reach live and fails on terminal errors;
  13. verifies exact revision and protocol headers from API and MCP /health;
  14. attests cloud readiness and fails if a supplied model credential did not become usable;
  15. attests social mention readiness when provider values were supplied;
  16. stores a redacted 30-day deployment evidence artifact.

Configure the GitHub Actions secret RENDER_API_KEY. Create it in the Render Dashboard for the workspace that owns these four services, then store it only under repository Settings > Secrets and variables > Actions. Never put the key in Render variables, workflow inputs, logs, issues, or Git. A missing key is a visible workflow failure, not a silent manual-deploy fallback. The key can deploy application services, so rotate it after suspected exposure. Creating, rotating, or revoking the credential is an explicit R3 access change; using the already-provisioned credential for the bounded exact-SHA application deploy is R2.

To enable hosted bounty drafting, also configure the repository Actions secret CLOUD_AGENT_API_KEY. It is passed only to the bounded deployment job, written only to agent-bounties-api, and redacted from evidence. If it is absent, the deployment still succeeds but /v1/cloud-agent/readiness remains unavailable and reports the missing credential explicitly; no local-model fallback runs.

To enable signed MoonPay checkout, configure both repository Actions secrets MOONPAY_PUBLISHABLE_KEY and MOONPAY_SECRET_KEY. Keep the repository Actions variable MOONPAY_ENVIRONMENT=sandbox while using pk_test_/sk_test_ keys; switch it to live only together with an approved pk_live_/sk_live_ pair. A partial or mode-mismatched pair fails before Render is changed. Run python scripts/check-moonpay-production.py --require-checkout after the exact revision is live; this validates URL generation and signature boundaries but does not complete a purchase or establish bounty funding.

To activate Farcaster ingestion, provision one approved Neynar signer owned by the Agent Bounties bot account. Store NEYNAR_API_KEY and NEYNAR_SIGNER_UUID as repository Actions secrets. Store NEYNAR_BOT_FID and NEYNAR_BOT_USERNAME as repository Actions variables. The controller registers the cast.created webhook filtered by that FID at https://api.agentbounties.app/v1/social/webhooks/neynar and installs the provider-generated webhook secret directly on Render. A partial provider configuration fails deployment instead of launching an unsigned or reply-less production listener.

The controller can be rehearsed without credentials:

python scripts\test_render_deploy_recovery.py -v
python scripts\check-render-blueprint.py

After the secret exists, workflow_dispatch can recover the latest successful CI revision reachable from main. An older successful SHA skips when a newer successful SHA exists, while a newer failed commit cannot suppress deployment of the last known-good revision. The scheduled Operational Control Loop stays read-only and continues to fail closed on revision skew; it does not possess the Render key. If current main is newer and failing, pass the latest successful 40-character SHA in the manual revision input.

If Render exhausts pipeline minutes, set the repository variable RENDER_DEPLOY_PAUSE_REASON=build_pipeline_minutes_exhausted. Every deployment mode then stops before touching Render. Restore bounded pipeline capacity or wait for the billing reset, delete the variable, and dispatch the latest successful main SHA.

Use deploy_only for runtime-only configuration after capacity is available:

  1. Read the exact SHA from the production /health revision header.
  2. Dispatch Render Deploy Recovery from current successful main.
  3. Select deploy_only and enter that production SHA as runtime_revision.
  4. Verify the workflow's exact health and readiness evidence.

deploy_only rejects a service whose current live artifact does not match the supplied SHA. It reuses that artifact, applies saved environment values, and does not build new code. It restarts API plus any service whose direct or shared environment changed, then requires the supplied SHA from /health and the exact leaderboard contracts from the live API. Render's branch label is recorded but is not artifact evidence. Render currently applies the workspace pipeline quota before both deployment modes, so deploy_only is not a quota bypass.

If the hosted operator credential may have been disclosed, dispatch this same exact-revision recovery workflow in deploy_only mode with rotate_operator_api_token=true. The replacement is generated only inside the runner, installed in the existing Render operator environment group, and never returned in logs or evidence. The API and MCP are both redeployed and attested; the evidence records only that rotation occurred.

The API and MCP services need the same DATABASE_URL, public URLs, factory, implementation, and Base RPC configuration. Canonical planners fail closed without Postgres and the active protocol addresses.

Recover a hosted API that returns 404

Run the deterministic public probe against the URL shown in Render, not a URL copied from an old issue or deployment record:

python scripts\diagnose_hosted_api.py `
  --base-url https://agent-bounties-api.onrender.com `
  --json-out target\operations\hosted-api-diagnosis.json `
  --md-out target\operations\hosted-api-diagnosis.md

The command checks DNS and the three public contracts required by the funding page: /health must return the exact body ok, while /v1/readiness/live-money and /v1/bounties/funding-feed must return JSON. A nonzero exit is expected until all three contracts pass. A 200 response with a static HTML body is classified as a route mismatch rather than healthy.

For an all-404 result, repair in this order:

  1. In Render, open Blueprints and apply the repository-root render.yaml from NSPG13/agent-bounties on branch main. If the Blueprint does not exist, use New > Blueprint, connect the repository, and apply it.
  2. Confirm the Blueprint created the Docker web service agent-bounties-api. Its settings must point at ./Dockerfile, use the repository root as Docker context, and use /health as the health check.
  3. Confirm the service variables are APP_PACKAGE=api and APP_BINARY=api. A worker or MCP binary on this hostname does not expose the funding routes. The repository-root Docker context must also copy every root-level asset compiled into a Rust binary. scripts/check-render-blueprint.py verifies those include_str! and include_bytes! dependencies, and the Containers workflow monitors every copied source directory so an application or asset change cannot bypass a production-image build.
  4. Copy the service's current onrender.com URL from Settings and rerun the diagnostic against it. Do not infer the hostname from the service name.
  5. If the Render URL passes, attach and verify api.bountyboard.global, set PUBLIC_BASE_URL=https://api.bountyboard.global and MCP_BASE_URL=https://mcp.bountyboard.global, then set the repository Actions variable PRODUCTION_API_BASE_URL to the verified API URL.
  6. Run python scripts\check-render-blueprint.py, dispatch Render Deploy Recovery for the latest successful main revision, and rerun the diagnostic plus the production smoke check.

The API starts with public Stripe Checkout disabled. After reachability is repaired, configure these API-service values in Render:

STRIPE_SECRET_KEY=sk_test_...                 # secret; use test mode first
STRIPE_WEBHOOK_SECRET=whsec_...               # secret; signed endpoint
STRIPE_API_BASE_URL=                           # empty means Stripe default
STRIPE_PAYMENT_METHOD_CONFIGURATION=          # optional PayPal-capable Dashboard configuration
OPERATOR_API_TOKEN=<generated secret>
ENABLE_STRIPE_LIVE_EXECUTION=true
ENABLE_STRIPE_PUBLIC_CHECKOUT=true
ALLOW_UNSIGNED_STRIPE_WEBHOOKS=false

Keep both enable flags false until the signed webhook, limits, and test-mode rehearsal in live-money-activation.md pass. Health or readiness success is reachability evidence only: it does not create funding, credit a balance, authorize payout, or prove a bounty is payable.

Environment

Non-secret protocol settings:

BASE_INDEXER_PROTOCOL=autonomous-v1
BASE_INDEXER_NETWORK=base-mainnet
BASE_INDEXER_START_BLOCK=<factory deployment block>
BASE_MAINNET_BOUNTY_FACTORY=<verified factory>
BASE_MAINNET_BOUNTY_IMPLEMENTATION=<verified implementation>
BASE_RECOVERY_RESERVED_BOUNTY_CONTRACTS=<comma-separated public incident contracts>
BASE_MAINNET_RPC_URL=<managed HTTPS RPC>
BASE_INDEXER_RPC_URL=<managed HTTPS RPC>
BASE_INDEXER_RETRY_INITIAL_SECONDS=5
BASE_INDEXER_RETRY_MAX_SECONDS=120
BASE_INDEXER_EXIT_AFTER_FAILURES=8
ENABLE_BASE_TX_BROADCAST=false

Use the corresponding BASE_SEPOLIA_* values for testnet. The worker accepts BASE_INDEXER_FACTORY_CONTRACT as an explicit override.

BASE_RECOVERY_RESERVED_BOUNTY_CONTRACTS is a public, temporary hosted-routing control. Every address must have a public incident record. Malformed values stop API and MCP startup; configured contracts remain visible in the full canonical feed but cannot appear as earning-ready work or verifier jobs.

Shared secrets belong in Render environment groups, never in Git:

  • DATABASE_URL,
  • managed RPC credentials,
  • optional OPERATOR_API_TOKEN for non-protocol administrative surfaces,
  • future Stripe secrets and verified webhook secret.

The hosted model credential is a direct API-service secret, not a shared environment-group value. This prevents MCP and the indexer from receiving it.

The separate RENDER_API_KEY belongs only in GitHub Actions and is never injected into an application container.

The autonomous contracts do not need a hosted private key, settlement signer, or owner key. Agents and relayers submit their own wallet transactions.

Contract Gates

Before any deployment:

  1. Run forge fmt --check, build, unit tests, 1,000+ fuzz runs, Slither or an equivalent static analyzer, and the Rust ABI/event suites.
  2. Deploy to Base Sepolia and execute funded pass, funded fail, claim timeout, verification timeout, pooled cancellation, refund, EOA authorization, ERC-1271 claim, and quorum settlement paths.
  3. Compare every Rust planner payload against independent Foundry cast vectors.
  4. Run the indexer from the deployment block and verify all four creation events and same-block funding are discovered.
  5. Publish the internal review and document accepted residual risks. Independent review is required before removing the low-value activation cap.
  6. Publish the exact source commit and deployment intent before mainnet signing.
  7. Ask for explicit action-time approval before broadcasting the mainnet deployment.

Testnet Deployment

Use a dedicated deployer wallet with only testnet funds. Do not paste a seed phrase or private key into chat, Git, shell history, browser storage, or committed files. The current exact addresses, constructor inputs, bytecode, and test-USDC seed are pinned in deployments/base-sepolia-sponsor-activation.json.

$env:Path = "$PWD\.tools\foundry;$env:Path"
forge test --root contracts\base-escrow --fuzz-runs 1000
python -m http.server 8879 --bind 127.0.0.1

Open http://127.0.0.1:8879/tools/base-sepolia-sponsor-activation.html in the browser profile containing the deployer wallet. The locked console verifies every action before requesting a wallet confirmation and supports safe resume after each confirmed component. See base-sepolia-runbook.md for regeneration, native USDC fork, post-deploy attestation, hosted configuration, and full-loop gates.

Mainnet Activation

Mainnet uses native USDC 0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913. The factory constructor has no other argument.

The low-value activation bundle is recorded in deployments/base-mainnet-activation.json. It is generated from the committed terms under bounties/autonomous-v1, the compiled Foundry artifacts, deployer 0x884834E884d6e93462655A2820140aD03E6747bC, and deployment nonce 4. The bundle contains unsigned factory deployment data, one aggregate 4 USDC approval, and four 1 USDC creation calls. Regenerate it and check the current on-chain deployer nonce immediately before signing; any nonce change requires a new bundle and predicted-address review.

The current post-and-complete seed batch is separate from that historical factory bootstrap. It is recorded in deployments/canonical-child-seeds-base-mainnet.json and depends on the exact verifier deployment artifact in deployments/canonical-child-verifier-base-mainnet-deployment.json. The locked local console at tools/autonomous-activation.html requires both artifacts, verifies factory and verifier bytecode/configuration, and supports atomic activation or bounded recovery from a partially confirmed sequential wallet flow. None of these files proves mainnet funding.

The exact existing-factory fork replay is recorded in docs/evidence/canonical-child-seeds-mainnet-fork-2026-07-13.json. It proves the verifier deployment and four funding calls execute together on a fork; its local transaction hashes are not mainnet evidence.

The repeatable Base-mainnet-fork result is recorded in docs/evidence/autonomous-v1-mainnet-fork-2026-07-11.json. That file proves rehearsal only. It is not live deployment, funding, or payout evidence.

The canonical factory deployment is recorded in docs/evidence/autonomous-v1-mainnet-deployment-2026-07-11.json. The four capped 1 USDC canary creations and their exact safe-block state are recorded in docs/evidence/autonomous-v1-mainnet-canaries-2026-07-11.json. Neither record proves completion or payout; only BountySettled does.

The permissionless deterministic verifier has a separate full-loop fork test in contracts/base-escrow/test/AgentBountyMainnetFork.t.sol. It forks current Base mainnet state, checks the exact deployed runtime hashes, creates and funds a canonical bounty with native USDC, claims from an independent address, submits hashes, mines the committed 16-bit proof, and settles from an unrelated relayer. It is opt-in so routine offline test runs do not depend on a public RPC:

$env:RUN_MAINNET_FORK = "true"
$env:BASE_MAINNET_RPC_URL = "https://your-base-mainnet-rpc"
cd contracts/base-escrow
forge test --match-contract AgentBountyMainnetForkTest `
  --match-test testCanonicalMainnetPermissionlessPaidLoop -vv

The reproducible run record is docs/evidence/permissionless-module-mainnet-fork-2026-07-11.json. The harness never broadcasts and fork settlement is not live payout evidence.

After a confirmed, verified deployment:

  1. Update deployments/base-mainnet.json with factory, implementation, transaction, block, deployer, and runtime hashes.
  2. Update site/protocol.json and the static discovery manifest from null to the same addresses and set status to active.
  3. Configure API, MCP, and worker environments.
  4. Set BASE_INDEXER_START_BLOCK to the factory deployment block on the first run.
  5. Deploy services and confirm indexer cursor/heartbeat progress.
  6. Run production smoke, post one low-value bounty, exercise pass and fail paths, and confirm the public feed never reports payment before BountySettled.

The worker scans the factory once per block range and canonical bounty clones in bounded multi-address batches. Cursor advancement happens only after event persistence.

Post-Deploy Checks

python scripts\check-site.py
python scripts\check-render-blueprint.py
cargo run -p cli -- production-smoke `
  --api-base-url https://api.agentbounties.app `
  --mcp-base-url https://mcp.agentbounties.app

Check:

  • /health, /llms.txt, OpenAPI, and discovery manifest,
  • protocol status and exact factory/implementation agreement,
  • canonical feed and verification-job feed,
  • terms and evidence persistence across restarts,
  • worker heartbeat and confirmed cursor,
  • no active legacy escrow endpoints or addresses,
  • no secret material in responses or logs.

Run the bounded operational controller after production smoke:

python scripts\self_heal.py observe `
  --policy ops\self-healing-policy.json `
  --api-url https://api.agentbounties.app `
  --mcp-url https://mcp.agentbounties.app `
  --expected-revision <deployed-git-sha> `
  --snapshot-out target\operations\snapshot.json `
  --plan-out target\operations\recovery-plan.json

API/MCP health failure is handled by Render's service supervisor. The indexer retries typed RPC/SQL transport failures from its persisted monotonic cursor with capped exponential backoff and exits after its bounded failure budget so the worker supervisor can replace the process. Integrity and unclassified failures halt ingestion after a redacted failed heartbeat. See self-healing-operations.md for SLOs, containment, and actions that automation is prohibited from taking.

Fiat Services

Stripe and PayPal are not autonomous-v1 settlement rails. Keep live execution disabled unless a separately reviewed fiat-to-USDC onramp is implemented with verified webhooks, compliance controls, idempotency, and exact canonical bounty funding reconciliation.