Desktop (macOS) — Developer Guide

Project Overview

OMI Desktop App for macOS (Swift)

Logs & Debugging

For difficult SwiftUI/AppKit runtime bugs—stale views, lost input, layout loops, jumps, beachballs, or fast-interaction failures—follow docs/swiftui-appkit-runtime-debugging.md.

Local App Logs

App log file: /private/tmp/omi.log (production). Each non-production launch writes to its own owner-only log; ask the running named bundle for its exact path with ./scripts/omi-ctl log-path rather than reading a shared dev log.

Release Health (Sentry)

Check errors in the latest (or specific) release using the sentry-release skill:

./scripts/sentry-release.sh              # new issues in latest version (default)
./scripts/sentry-release.sh --version X  # specific version
./scripts/sentry-release.sh --all        # include carryover issues
./scripts/sentry-release.sh --quota      # billing/quota status

Run the script with --help for the full option list.

User Issue Investigation

When debugging issues for a specific user, check Sentry dashboard for crashes and PostHog for events.

Product analytics integrity

A desktop chat query starts after local concurrency/quota preflight and must emit exactly one terminal outcome: completed, failed, or cancelled. Intentional Stop and supersession are cancellations, never errors.
A physical voice shortcut turn emits one start and one terminal outcome from the coordinator. Full-answer duration ends at playback drain; a later journal failure does not rewrite a delivered response as missing. Intentional and too-short endings are excluded from response-failure rates.
Query latency ends when the final answer is visible. Persistence, title generation, and other post-answer work have their own reliability signals and must not inflate user-visible query duration.
Product authority is independent from telemetry. Revoked or timed-out turns cannot apply late callbacks/results or persist a late response even if analytics is disabled or refactored.
PostHog receives bounded dimensions and shape metadata only. Never send raw prompts, responses, notification/window titles, filesystem paths, or exception messages. Keep diagnostic detail in the private local log and Sentry.
Production QueryTracer output is shape-only and stored under a 0700 directory in 0600 files. Full prompt/response/tool content is a deliberate non-production debugging capability only.

Fallback / resilience telemetry

Provider/mode switches and fail-open paths must call DesktopDiagnosticsManager.recordFallback(area:from:to:reason:outcome:) (PostHog desktop_health_event / fallback_triggered) or Rust fallback::record_fallback. Same field contract as root AGENTS.md → Fallback / resilience telemetry. Do not invent new health-event enum cases or product “Recording Error” events for successful heals (outcome=recovered).

Gemini Live has no safe mid-session system role. Background agent/card text stays on its canonical tool or visible UI surface and must never use Gemini's realtime user-input wire.

Repository

This is the desktop/macos/ subfolder of the OMI monorepo (BasedHardware/omi)
macOS Swift app lives here; its shared Python desktop backend lives under ../../backend

Release Pipeline

Beta candidates ship on the hourly release train: desktop_auto_release.yml runs on an hourly schedule (plus manual workflow_dispatch retries — there is no push trigger). Each run tags the newest releasable desktop/macos/**/codemagic.yaml change since the last tag, coalescing every merge in the window and binding the immutable tag to the Codemagic config that builds it. A candidate reaches Beta automatically after Codemagic finishes the signed release build:

GitHub Actions (desktop_auto_release.yml) — the planner auto-increments the version and pushes a timestamped annotated v*-macos tag. Every invocation passes --min-tag-interval-seconds 3600, so scheduled and manual retries create at most one candidate per hour, measured from the previous candidate's GitHub-release createdAt after publication or its annotated-tag creation time while still building. A ~60s quiet window (AUTO_RELEASE_QUIET_SECONDS) coalesces near-simultaneous merges; the one-active-release fence (Codemagic build status on the latest tag) admits one candidate at a time. GitHub compile queues do not gate tagging: the tag stays on the exact newest source, or its mechanically verified changelog-only child, while later macOS merges remain queued for the next hourly candidate. The workflow immediately API-dispatches exactly one same-tag omi-desktop-swift-release build, verifies its immutable source identity, and retains JSON intake evidence; retries reuse an existing exact-tag build instead of duplicating it.
Codemagic (codemagic.yaml, workflow omi-desktop-swift-release) — API-dispatched with the immutable tag, runs on Mac mini M4 and owns compile/release admission:
- Builds the universal app and dSYM, UUID-checks and attempts to upload symbols to Sentry, and always publishes the verified dSYM with the release so a Sentry credential outage is repairable without blocking the signed candidate
- Signs with Developer ID, notarizes with Apple
- Creates DMG + Sparkle ZIP
- Runs scripts/smoke-signed-desktop-artifact.sh (signed app, Sparkle ZIP, DMG) before publishing, with a mandatory in-app Keychain write/read/delete canary
- Publishes an immutable GitHub candidate with stable and Beta signed-smoke evidence
- Calls the internal-only Beta admission endpoint with the exact tag; the backend independently verifies the merged tag, GitHub asset digests, and Beta signed-smoke evidence, then atomically registers the immutable manifest and advances the explicit Beta pointer

The former self-hosted T2 qualification and post-qualification promotion workflows are not part of the release path. Local T2 and fault suites remain available as engineering QA tools, but they do not gate Beta visibility. If Codemagic exhausts its bounded promotion retries, the candidate remains non-live; recover with desktop_recover_beta.yml using the exact tag, confirm=recover-beta, and a reason.

The shared Python backend must contain the manifest/pointer endpoints before the first beta promotion. Deploy it separately with gcp_backend.yml; desktop_backend_auto_dev.yml owns development Python desktop-backend delivery, while desktop_backend_prod.yml and desktop_backend_recover_prod.yml own protected production delivery and retained-revision recovery. Merging desktop code does not deploy either production backend. Static GCS/CDN feed ownership remains follow-up work and is not the channel source of truth.

Signed artifact smoke scope:

Always-on release audit covers bundle identity, version/tag alignment, signing/Keychain entitlements, Sparkle metadata, backend URL leakage, helper/runtime packaging, artifact readability, and local storage package surface.
Stable artifacts are fixed to production Python/Rust services. The separately-installable Beta artifact is fixed to development Python/Rust services, while desktop-login OAuth plus Firebase Auth/Firestore remain production; signed smoke proves the artifact routing. Exercise a real human OAuth sign-in separately before broad rollout.
Codemagic uploads stable and Beta smoke results with artifact digests and completed checks; the backend fetches both results concurrently and compares them to the exact immutable release assets before changing channels.
The synthetic --auth-storage-canary is mandatory before beta publication and runs inside the exact signed app without real credentials. Optional broader live probes (--launch --network --auth --chat --permissions --storage) require an isolated release runner and explicit canary env vars; production-bundle launch is fail-closed unless OMI_SIGNED_ARTIFACT_SMOKE_ALLOW_PRODUCTION_LAUNCH=1, and --auth requires OMI_SIGNED_ARTIFACT_SMOKE_AUTH_PROOF_COMMAND to prove app-level persistence rather than a raw bearer-token curl.
Artifact creation and user visibility are split: create/upload the immutable candidate first, then advance Beta visibility only after its digest-matched signed smoke passes.
Automatic Beta is fail-closed: any signed-smoke, reservation, digest, manifest, admission-generation, or pointer failure leaves the candidate non-live. Operators pause/resume only through the ADMIN_KEY-protected backend admission control; workflow variables are not pause authority. When a served Beta is concretely broken, desktop_rollback_beta.yml and desktop_breakglass_rollout_beta.yml retain their audited incident-only behavior. Rollback does not downgrade already-updated clients; follow it with a higher-build repair.

Stable is manual:

Codemagic never promotes Stable. desktop_promote_prod.yml remains workflow_dispatch only and protected by the prod environment.
Run it with the current Beta release_tag and confirm=promote-stable. It reads and compare-and-swaps the current Stable pointer itself, advances only that pointer, updates the existing legacy/static bridges, and verifies hashes and feed output.
The Python desktop-backend deployment workflows remain independent from desktop release promotion. Stable promotion checks live desktop chat-contract compatibility but does not deploy the backend. Do not manually edit release visibility or pointers outside the promotion workflow.

Codemagic CLI & API:

Token: $CODEMAGIC_API_TOKEN (set in ~/.zshrc)
App ID: 66c95e6ec76853c447b8bcbb
List builds: curl -s -H "x-auth-token: $CODEMAGIC_API_TOKEN" "https://api.codemagic.io/builds?appId=66c95e6ec76853c447b8bcbb" | python3 -c "import json,sys; [print(f\"{b.get('status','?'):12} tag={b.get('tag','-'):30} start={(b.get('startedAt') or '-')[:19]}\") for b in json.load(sys.stdin).get('builds',[])[:5]]"

Firebase Connection

Use the /firebase command if your agent provides it.

Quick connect:

cd ../backend && source venv/bin/activate && python3 -c "
import firebase_admin
from firebase_admin import credentials, firestore, auth
cred = credentials.Certificate('google-credentials.json')
try: firebase_admin.initialize_app(cred)
except ValueError: pass
db = firestore.client()
print('Connected to Firebase: based-hardware')
"

Module Layout (SwiftPM)

Desktop/Package.swift is incrementally splitting the monolithic executable into library targets with enforced dependency edges:

OmiTheme — shared colors, typography, chrome (Sources/Theme/)
OmiWAL — write-ahead log model + coordinator (Sources/OmiWAL/)
OmiSupport — shared desktop runtime helpers (Sources/OmiSupport/, e.g. DesktopLocalProfile and Dictionary(lastWriteWins:))

Rewind/Core/ remains in the executable target for now — it still references main-app types (TaskActionItem, PowerMonitor, etc.) and needs a shared-models carve-out first.

Do not add new .swift files directly under Desktop/Sources/. Place new code in a feature directory (Onboarding/, MainWindow/, Chat/, etc.). CI enforces this via scripts/check-sources-root-layout.py.

When carving out additional leaf modules, prefer bottom-up order (models and storage before UI) and wire import + public on the extracted target's API.

Bundled resources

.process("Resources") caches its manifest: after adding a file under Sources/Resources/, touch Desktop/Package.swift or the build silently omits it. It may flatten subdirectories — search both roots (OmiSoundAssetLocator), never Bundle.module. Cinematic audio is generated: scripts/make-onboarding-sounds.py.

Swift Formatting

Swift formatting uses a pinned swift-format binary (release 602.0.0 at commit 62eaad2), bootstrapped from source via scripts/swift-format-wrapper.sh. The config lives at Desktop/.swift-format (2-space indent, 120-column limit). Generated sources under Desktop/Sources/Generated/ are excluded from the formatter scope. Bootstrap once: ./scripts/swift-format-wrapper.sh bootstrap. Lint the full scope: ./scripts/swift-format-wrapper.sh lint -r $(./scripts/swift-format-wrapper.sh scope).

SwiftLint

SwiftLint safety rules run as an explicit macOS manifest check (not a SwiftPM build-tool plugin) through scripts/swiftlint-wrapper.sh. The wrapper pins the upstream 0.65.0 universal macOS release artifact by SHA-256 and caches the verified binary under ~/.cache/omi-swiftlint; use ./scripts/swiftlint-wrapper.sh lint to run the full configured scope. Generated sources and test fixtures remain excluded and the committed baseline is down-only. SwiftLint baseline locations are absolute, so the wrapper materializes a temporary baseline rooted at the current checkout before linting; do not hand-edit those paths to match a specific machine.

Synchronous state-machine callbacks

A reducer transition is atomic through model assignment, effect delivery, UI projection, and snapshot publication. A callback may request another event, but it must not recursively reduce against a half-published transition.
Coordinators with synchronous effect/snapshot callbacks drain nested events through a FIFO, non-reentrant queue. Do not fix recursion with one-off boolean suppression or by dispatching after an arbitrary delay.
Tests for callback-driven machines must synchronously enqueue from both an effect callback and an observer/snapshot callback, assert callback depth stays one, and assert the resulting event order.

Collection safety

Never use Dictionary(uniqueKeysWithValues:) for API responses, decoded persistence, runtime projections, or any other data whose key uniqueness is not enforced by the Swift type system. A duplicate key traps and terminates the process.
Use Dictionary(lastWriteWins:) from OmiSupport when the newest record in input order is authoritative. Use another explicit non-trapping merge policy when the domain requires different semantics.
A raw trapping initializer is allowed only for a statically proven uniqueness contract, with a local reason: // omi-collection-safety: static-unique-keys -- <why the type guarantees uniqueness>. Runtime validation, backend expectations, and “should be unique” are not static contracts.
Run python3 scripts/check_desktop_test_quality.py after changing Swift collection construction.

Swift test quality

Behavior fixes require tests that call the production API and assert outcomes. Reading a production .swift file and asserting that it contains a function name or implementation string is not behavioral coverage.
Source inspection is reserved for narrow forbidden-pattern or static wiring tripwires. New tripwires must carry a local reason: // omi-test-quality: source-inspection -- static contract: <what cannot be expressed behaviorally>. The tripwire supplements rather than replaces behavioral coverage.
Do not add wall-clock sleeps to unit tests. Inject a Clock/sleeper, drive a callback/continuation, or await a deterministic state signal. An unavoidable real-scheduler integration wait needs // omi-test-quality: wall-clock-wait -- <why injection cannot test this boundary>.
python3 scripts/check_desktop_test_quality.py ratchets both legacy source-inspection sites and wall-clock waits; its baselines may only decrease.

Key Architecture Notes

Authentication

Firebase Auth with Apple/Google Sign-In
Desktop apps should use backend OAuth flow: /v1/auth/authorize
Apple Services ID: me.omi.web (shared across all apps)
iOS apps use native Sign-In, Desktop uses backend OAuth + custom token
Session death is owned by AuthSessionCoordinator (INV-AUTH-1); use invalidateSession for expired/revoked Firebase creds, not nuclear signOut().

Session 401 vs BYOK/provider 401

Failure class	Owner	Action on 401 after forced refresh
Firebase session token (default API `Authorization`)	`AuthSessionCoordinator`	`invalidateSession` → Sign-in CTA
BYOK provider key on request	`CredentialHealthManager`	Suppress/mark provider unhealthy; do not invalidate Firebase session
Realtime/voice managed lane	`CredentialHealthManager` + hub UX	`requiresLogin` only when session mint fails after refresh
Background poll with `RequestAuthPolicy.sessionPreserving`	Caller	Throw `.unauthorized`; no session invalidation
`DesktopLocalProfile` harness	Auth emulator bootstrap	Re-bootstrap emulator session; no prod invalidation side effects

Database Structure

Firestore (based-hardware): User data, conversations, action items
Redis: Caching
Typesense: Search

Screen activity sync rollout

screen_activity_lossless_sync enables durable per-row delivery, five-minute (app, window) compaction, and bounded embedding recovery. Production-family bundles stay on the legacy path until that PostHog flag is true; non-production bundles dogfood it by default and OMI_FORCE_LOSSLESS_SCREEN_SYNC=0 disables it locally.
OCR-bearing rows sync independently from embeddings. Embeddings are an optional later projection and must never gate capture, OCR, or text delivery.
Firestore screen-activity timestamps use the lexicographically sortable UTC form yyyy-MM-dd HH:mm:ss.SSS. The backend normalizes ISO-8601 input before storage.

Feature-flag authority

Bundle vs PostHog vs runtime_env is catalogued in backend/docs/feature-flag-registry.md. Editing that file does not turn a feature on. Do not target Beta vs stable via PostHog person update_channel.

User Subcollections (Firestore)

users/{uid}/conversations - Has source field (omi, desktop, phone, etc.)
users/{uid}/action_items - Tasks (no platform tracking)
users/{uid}/fcm_tokens - Token ID prefix = platform (ios_, android_, macos_)
users/{uid}/memories - Extracted memories

User MCP Servers & Skills (Apps page)

User-managed MCP servers (~/.omi/mcp.json, incl. native OAuth) and skills (~/.omi/skills/<slug>/SKILL.md), fully local and fail-open. Contract and runtime wiring: .github/agent-docs/desktop-user-extensions.md.

Known Limitations

Firestore has no collection group indexes for source field
Counting users by platform requires iterating all users (slow)
Apple Sign-In: Only one Services ID per Firebase project

Development Workflow

Building & Running

No Xcode project — this is a Swift Package Manager project
Build command: xcrun swift build -c debug --package-path Desktop (the xcrun prefix is required to match the SDK version)
run.sh prepends the native Homebrew prefix (/opt/homebrew/bin on Apple Silicon or /usr/local/bin on Intel), followed by the other prefix when present, because agent and launchd shells may not inherit Homebrew's PATH. This keeps pkg-config and other build tools discoverable without requiring a machine-wide shell profile change.
Full dev run: ./run.sh — builds Swift app, starts Python backend, starts Cloudflare tunnel, launches app
Fast default dev run: after one successful full named-bundle launch, ordinary Swift-only ./run.sh calls reuse the installed bundle. The fast lane runs incremental SwiftPM, atomically replaces the executable and current desktop API URL, re-signs the app, and relaunches without copying/re-signing static agent/framework assets or resetting LaunchServices/auth. It re-syncs the curated settings allowlist from the resolved settings authority (production "Omi" when installed, else Omi Dev) before every named-bundle launch so hotkeys and other launch preferences cannot go stale; use OMI_SKIP_SETTINGS_SEED=1 only when intentionally testing bundle-local settings. Named local profiles are eligible: their current disposable .env is refreshed on each patch and is never cached in the bundle fingerprint. Package metadata, resources, agent/runtime inputs, entitlements, persistent launch configuration, and an installed bundle whose agent runtime payload is incomplete (incomplete_runtime_payload; scripts/agent-runtime-payload.sh) automatically take the full path. Force that path with ./run.sh --full or OMI_FORCE_FULL_BUNDLE=1. OMI_SCAN_STALE_BUNDLES=1 is an explicit stale-LaunchServices recovery scan; do not enable it in the normal loop.
Focused feedback loop: ./scripts/dev-feedback.py --once|--watch swift '<XCTest filter>' or ... python '<pytest path>' runs exactly the regression you selected and reports each iteration time. It watches only the matching component inputs, keeps watching after a failure, and never replaces the full component suite. A filter that matches no tests fails the iteration (swift test --filter exits 0 on zero matches), so a renamed or mistyped filter can never read as PASS. Pre-push deliberately adds only xcrun swift build -c debug; never promote it to the full pinned-Xcode suite or release compile, because that push-time budget belongs to CI.
Swift suite throughput: Local suites default to four workers. CI uses two workers only because each gets a copy-on-write SwiftPM scratch directory and an isolated Foundation runtime home (preferences, Application Support, caches, and temporary files). Do not raise it without evidence that both build and runtime state remain isolated. Set OMI_SWIFT_TEST_SUITE_WORKERS=1 to diagnose concurrency failures.
Local Python backend: direct ./run.sh development reuses a healthy backend that this worktree owns when Python source/config have not changed. Sync dependencies with cd ../../backend && ./scripts/sync-python-deps.sh before the first local launch.
Agent runtime preparation cache: local ./run.sh calls reuse validated agent packaging from the worktree-local .harness/agent-runtime cache when source, locks, preparation logic, pinned runtime, mode, OS/architecture, Node/npm versions, and every file copied from the prepared runtime are unchanged. Hits verify the complete agent dist, both packaged dependency trees, their symlinks, and staged Node; working agent/node_modules is not hashed. The script logs Cache HIT, MISS, or BYPASS; hits preserve output mtimes but spend roughly a second on a warm local filesystem hashing the packaged outputs for integrity (hardware/filesystem dependent). CI and --skip-npm always bypass the stamp. Set OMI_AGENT_RUNTIME_FORCE_REBUILD=1 for an explicit local rebuild. Do not copy this cache between worktrees or treat it as a release artifact. The checksum-verified universal Node archives are separately shared at ~/Library/Caches/OmiDesktop/node-archives (override with OMI_AGENT_RUNTIME_ARCHIVE_CACHE_DIR), so fresh linked worktrees reuse the download but still validate it before staging.
Release builds: Handled entirely by Codemagic CI (no local release script needed)
DO NOT use bare swift build — it will fail with SDK version mismatch
DO NOT use xcodebuild — there is no .xcodeproj
DO NOT launch from build/ or hand-copy binaries into a bundle — always ./run.sh. It installs to /Applications/, signs, and registers with LaunchServices; build/ binaries go stale after a permission restart.
Code signing: local entitlements key on the identity's Team ID, not its name. Never use OMI_ALLOW_ADHOC_SIGN=1 to fix a launch failure — it kills that bundle's Screen Recording grant. docs/local-code-signing.md
DO NOT kill, delete, or interfere with running "Omi", "omi", or "Omi Beta" app bundles — these are production/release installs the user relies on

App Names & Build Artifacts

./run.sh builds "Omi Dev" → installs to /Applications/Omi Dev.app (bundle ID: com.omi.desktop-dev)
"Omi" stable (bundle ID: com.omi.computer-macos) and "Omi Beta" (bundle ID: com.omi.computer-macos.beta, isolated "Omi Beta" storage root, runs side-by-side with stable) are built by Codemagic CI only
To check which app is currently running: ps aux | grep "Omi"

Testing with Named Bundles

When the user asks to test a feature or bug fix, always create a separate named bundle so it can run side-by-side with the existing dev/prod apps:

OMI_APP_NAME="omi-fix-rewind" ./run.sh

This creates /Applications/omi-fix-rewind.app with bundle ID com.omi.omi-fix-rewind, completely independent of "Omi Dev" and "Omi Beta". Name it after the feature/bug being tested. The user can then run multiple test builds simultaneously without interfering with each other or the production app.

Build-lock invariant: ./run.sh locks per worktree (repo-root .dev/run-sh-build.lock.d), through build→install→seed→open, then releases before the long-running wait. Parallel worktrees must not block each other. Two named-bundle builds in the same worktree still serialize (shared Desktop/.build/). Do not reuse the same explicit OMI_APP_NAME across worktrees — /Applications/$APP_NAME.app is machine-global and not cross-locked.

Rules:

NEVER use the default ./run.sh (which overwrites "Omi Dev") when testing a specific feature — always set OMI_APP_NAME
ALWAYS prefix the name with omi- (e.g., omi-fix-rewind, omi-6512-polling, omi-vision-test) so named bundles are visually grouped in /Applications/ alongside "Omi Dev" and "Omi Beta"
Use short names: the name sets the app name and bundle ID suffix.
The named bundle gets its own permissions and writable database. A full ./run.sh install auto-seeds auth/onboarding and a one-time consistent Rewind snapshot from the shared local profile; every full or fast named-bundle launch mirrors the curated settings allowlist — including both hotkeys — from the resolved settings authority: OMI_SETTINGS_SEED_SOURCE if set (fail-closed when that domain is missing), else production "Omi" (com.omi.computer-macos) when installed, else "Omi Dev". Set OMI_SKIP_REWIND_SEED=1 to start with an empty Rewind profile or OMI_SKIP_SETTINGS_SEED=1 to preserve intentional bundle-local settings.
JIT QA: runbook.
To connect agent-swift: agent-swift connect --bundle-id com.omi.omi-fix-rewind
Skip the web login: sign into "Omi Dev" once; named bundles launched by ./run.sh clone that session before launch, falling back to the production "Omi" session when Omi Dev's is missing (OMI_AUTH_DUMP_SOURCE=<bundle-id> pins a source explicitly).
Jump to a screen without clicking: the automation bridge auto-enables on non-prod bundles — ./scripts/omi-ctl navigate <screen> (e.g. rewind, memories, settings rewind). See "Fast-Path for Local Iteration" in e2e/SKILL.md.
Named/dev bundles default to the development Python backends unless an explicit launch URL overrides them. Before QA, run ./scripts/omi-ctl health; its unauthenticated identity payload reports the resolved backend environment/URLs plus the agent-runtime handshake state, negotiated protocol version, packaged runtime version, and expected protocol. A protocol-compatible runtime that omits a required capability is rejected at startup; health never reports the expected protocol as if it were negotiated.
Run ./scripts/agent-logic-harness.sh --cross-surface-smoke before building a QA bundle. This is the compact Swift/Node/Python contract gate; reserve full component suites and the live continuity gauntlet for PR readiness.

Run Variants & Parallel Worktrees

./run.sh --yolo — quick start against the dev backend, no local services. OMI_SKIP_BACKEND=1 — app only, remote backend via OMI_DESKTOP_API_URL. OMI_SKIP_TUNNEL=1 — no Cloudflare tunnel.
Parallel worktrees auto-isolate. scripts/dev-instance.sh derives a unique instance from each linked git worktree, so run.sh (and backend/scripts/dev-serve.sh) pick per-worktree ports (desktop 10201+, Python 8080+, automation 47777+) and bundle name (omi-<worktree>). Kills are pidfile-scoped, and a taken port fails loud instead of clobbering. The primary checkout is unchanged (Omi Dev, 10201/8080/47777). Override any of OMI_INSTANCE / PORT / PYTHON_PORT / OMI_AUTOMATION_PORT / OMI_APP_NAME to opt out.
Omi Dev is the canonical shared development profile (reusable permissions, default auth seed source; settings authority only on machines without production "Omi" installed). To rebuild the real Omi Dev (com.omi.desktop-dev) from a linked worktree, pass the explicit override OMI_APP_NAME="Omi Dev" ./run.sh — auto-isolation would otherwise derive an omi-<worktree> named bundle, and the explicit name resolves back to the shared dev bundle id.
Local Python backend (per-worktree port): cd backend && ./scripts/dev-serve.sh.

Self-Testing the App (agents)

Hard rule: you may not ask the user to verify a feature you have not actually exercised yourself. Compiling, "looks correct from the code", or "scroll down to see it" are not verification. If the obvious path is blocked (permission, focus, missing tool), try a long sequence of alternatives before involving the user — extend the bridge with a new action, add a temporary in-process hook, search the web for a workaround, grant the missing permission yourself if you can, write a tiny standalone harness. Roughly: spend ten serious attempts across different approaches before you escalate. Asking the user is the last move, not the first.

Fast path (skips web login and sidebar click-through):

Build + launch a named bundle (see Testing with Named Bundles above). ./run.sh auto-clones Omi Dev auth/onboarding plus common shortcuts/settings before launch. Manual seeding:

./scripts/omi-auth-dump.sh                                  # capture the Omi Dev session
./scripts/omi-auth-seed.sh com.omi.omi-<feature> \
  tmp/desktop-auth.json "/Applications/omi-<feature>.app"   # clears stale Keychain; UD→KC migrate
./scripts/omi-settings-seed.sh com.omi.omi-<feature>        # replay shortcuts/settings

Prefer the local bridge — it never touches the cursor. It calls the app's real code in-process (no synthetic mouse events). Use it before reaching for agent-swift click/cliclick/computer-use. Auto-enables on non-prod bundles; run several at once via distinct OMI_AUTOMATION_PORT (default 47777). Navigation stays backgrounded unless --show is passed.
- ./scripts/omi-ctl state — app-state snapshot (selected tab, auth, onboarding).
- ./scripts/omi-ctl navigate <screen> [settings-section] — jump straight to a screen in ~150ms (omi-ctl screens lists targets).
- ./scripts/omi-ctl actions then ./scripts/omi-ctl action <name> [k=v …] — semantic actions (e.g. refresh_all_data). Add new ones in DesktopAutomationActionRegistry. See e2e/SKILL.md §2b.
- agent-swift only for UI the bridge can't reach yet (click moves the cursor).
Read logs to confirm behavior: app + chat bridge in the exact path from ./scripts/omi-ctl log-path (named dev bundles) or /private/tmp/omi.log (production); ./run.sh prints the isolated local Python desktop-backend log path at launch; per-user issues in Sentry/PostHog.
Verify the actual behavior, not just that the app launched — exercise the feature and check the logs/UI reflect the change.

Default agent development loop

Edit or diagnose: run the smallest relevant unit/static harness. For repeated saves, start ./scripts/dev-feedback.py --watch swift '<filter>' or ... python '<pytest path>'; do not launch the app only to obtain compile evidence.
Swift/UI behavior: reuse the existing named bundle with OMI_APP_NAME=omi-<feature> ./run.sh --yolo --fast-only; add --no-wait only with a harness/external backend, then use the local bridge (omi-ctl action, state, or a semantic snapshot) to assert the changed behavior.
Package boundary: use ./run.sh --full only for the first named launch, resource/entitlement/package/runtime input changes, or when --fast-only reports an expected fingerprint mismatch.
QA, commit, and PR readiness: run ./scripts/omi-macos-dev doctor, exercise the real user-facing path, then run the appropriate full component/PR contract.

omi-macos-dev defaults to bounded JSON summaries so an agent can safely inspect a busy machine. Pass --verbose to the specific command for path-level records (for example, clean plan --verbose); cleanup always requires the exact current plan hash. The normal 14-day retention window can be deliberately bypassed with --older-than 0 only when the operator has explicitly approved immediate cleanup.

Never ask a user to test an unexercised path. A fast named-bundle launch plus a semantic bridge assertion is valid inner-loop evidence; a clean full bundle is release/QA evidence.

After Implementing Changes

xcrun swift build is for compile checks only — it does NOT start the backend
Voice-path verification means a natural authenticated PTT turn on a named bundle — signed-out, forced-transcript, or reducer-only runs do not count; provider mint or payload changes must also show the deploy-inline provider probe.
When the user says "test it", use the test-local skill to build, run, and verify via macOS automation

macOS Version Compatibility

The deployment floor is .macOS("14.0") in Desktop/Package.swift. Every change must work on every supported macOS version from that floor up.
Never call an API newer than the floor unguarded: wrap it in if #available(macOS XX, *) and give the else branch a working fallback (degrade the feature, don't blank it). Example: System Audio capture gates on #available(macOS 14.4, *) and hides cleanly below it.
Version-dependent system facts (renamed apps, moved paths, changed defaults) get an explicit mapping with the old value still handled — stored user data may predate the change (example: AppIconCache.renamedApps maps "System Preferences" → "System Settings").
Raising the deployment floor or dropping a fallback is a product decision — never do it as a side effect of another change.

Open-Source Merge Hygiene

Before starting and before committing, git fetch origin && git rebase origin/main (or merge) — other contributors land changes continuously; never review your diff against a stale base.
Keep diffs surgical: touch only lines your change needs. No drive-by reformatting, renames, or import reshuffles in files others may have in-flight PRs against.
After rebasing onto new upstream work, re-run the test suites for every file you touched and every file the rebase brought in that overlaps your change; a clean build alone is not revision.
If your change modifies shared surfaces (Theme tokens, SettingsSection, bridge actions, INV-* contract files), grep for all usages — including tests and e2e flows — and update them in the same commit so concurrent contributors inherit a consistent tree.

Agent Logic Harness

When touching desktop agent runtime, floating agent pills, realtime hub, PTT, or pi-mono-extension, run the focused harness before broader checks:

cd desktop/macos && ./scripts/agent-logic-harness.sh

It is self-driving for agents: it runs the risky Swift lifecycle/state tests, focused agent runtime tests, exact pi-mono-extension package tests, and prints per-step runtime. Use --swift-only, --node-only, or --skip-install only when narrowing a failure.

Chat Continuity Write-Path Contract (INV-6)

Invariant: Main Chat, Home chat, and floating/notch chat are one timeline over one ChatProvider (historyChatProvider). Kernel main_chat turns are the durable source of truth; journal acceptance publishes the immediate pending projection, and UI must never append a pre-journal turn.

Rules (fail the PR if any break):

Single provider + floating viewport — floating presentation is chrome + a viewport cursor (FloatingChatViewport message ids / clientTurnId) over ChatProvider.messages. It must not own a second durable transcript array (chatHistory of ChatMessage copies is forbidden).
Single turn_recorded UI apply gate — only KernelTurnProjection on ChatProvider.mainInstance (historyChatProvider) may attach the runtime turn handler (one replaceable slot). Speculative warm and other surfaces must reuse mainInstance; never construct a second ChatProvider() that calls attachClient / setTurnRecordedHandler on the shared runtime.
One idempotency key per logical turn — call recordJournalExchange (or the corresponding kernel control RPC) with one opaque continuity key and await acceptance before binding a visible row. Direct-control spawn receipts already materialize their exchange; refresh that journal instead of issuing a second write. Never dedupe by assistant/user text.
Kernel apply is idempotent — KernelTurnProjection upserts only by the canonical turn ID published by ordered journal replay. Rejection must leave no visible row, and replay/acknowledgement must replace rather than append.
Cross-surface agent identity is structured — agentSpawn / agentCompletion content blocks (plus tool-block spawnedAgentID / sessionId / runId lines) are authoritative. Persist structured blocks through the kernel journal/outbox so they survive reload; kernel apply still materializes agentCompletion from bracket text for legacy rows. Legacy [Background agent id=…] bracket text remains dual-read only. Do not invent new free-text formats; extend the schema + tests together. Proactive notifications use continuity key notification:<uuid> (origin proactive_notification) and enter the notification-to-chat cache only after journal acceptance; do not reintroduce local timeline append paths.
Pill cache is derived — open-by-id hydrates from kernel (listFloatingAgentPills / listAgentSessions / inspectAgentRun) when the in-memory pill is missing; refresh-on-miss is a fast path only. Success = resolvable agent after hydrate. Do not keep a second durable pill store.
Snapshots are aliases — automationFloatingChatSnapshot == automationChatSnapshot / automationMainChatSnapshot over the same messages; no surface-specific transcript filter.
Resources live on the producing message — artifacts attach to the ChatMessage that produced them (stage/promote keeps resources on that id). UI must not invent a standalone artifact-only turn. Floating/notch resource strips bind message.displayResources on viewport-derived messages only (never flatMap the whole provider timeline). Aggregate strips must filter with ChatContinuityInvariants.resourcesBelongingToMessages / FloatingControlBarState.viewportDisplayResources.
Agent card/list preview = prompt/objective — collapsed header / list subtitle uses ChatContinuityInvariants.agentPreviewText(prompt:output:) (prompt wins; output is expanded-body only). Do not put raw completion output in the one-line preview.
Forbidden dual-write patterns — never: construct ChatProvider() for speculative warm (use ChatProvider.mainInstance); add addTurnRecordedHandler / multi-handler append APIs; introduce suppressNextRecordedTurn; store @Published var chatHistory of ChatMessage copies on FloatingControlBarState.
Tests — continuity behavior changes require a hermetic behavioral test (call projection/provider APIs, assert message counts/IDs). Source-string greps for function names are not continuity coverage (forbidden-pattern tripwires are the exception). Live gauntlet/stress are gates, not substitutes for hermetic tests.

Continuity PR Definition of Done (INV-6)

A PR that touches chat write-path, kernel projection, floating viewport, agent timeline identity/open, or pill projection is incomplete until:

Contract still true — INV-6 rules above hold after the change (or are updated in the same PR with a matching behavioral test).
Hermetic behavioral test for the invariant touched (stage/promote, snapshot alias, structured identity, open-by-id hydrate, viewport derive / restore, resources-on-message, agent preview text). Not a source grep (except forbidden-pattern tripwires).
./scripts/agent-logic-harness.sh green (includes KernelTurnRecordedProjectionTests, ChatTimelineContinuityTests, FloatingControlBarStateTests, RuntimeOwnerIdentityTests in the Swift focus filter).

Write-path / cross-surface changes: run a named-bundle continuity gauntlet and note evidence in the PR:

cd desktop/macos && OMI_APP_NAME=omi-gauntlet OMI_SKIP_TUNNEL=1 ./run.sh
# run.sh seeds auth after install (UD tokens → app Keychain migrate). Manual reseed:
# ./scripts/omi-auth-seed.sh com.omi.omi-gauntlet tmp/desktop-auth.json "/Applications/omi-gauntlet.app"
./scripts/agent-continuity-gauntlet.sh --suite continuity --bundle-id com.omi.omi-gauntlet
./scripts/check-gauntlet-evidence-at-head.sh

CI only runs gauntlet --self-check (wiring). Live suite is a PR/RC gate, not PR CI. Do not assert exact assistant wording.

Hermetic e2e only if a bridge action/surface contract changed. Do not expand flow covers: lists as fake continuity coverage.
No second message store / no new free-text identity format / no suppressNextRecordedTurn-style dual-write bandage.
Changelog fragment only if user-visible.

Gauntlet / stress gate policy

CI: agent-continuity-gauntlet.sh --self-check only (via desktop-core / agent-logic harness). Never require live LLM in PR CI.
Prompt / gateway changes: --suite prompts on a named omi-* bundle; P4 requires a completed public-web lookup with a source URL and fails on provider tool-choice incompatibilities. Continuity PRs / RC: --suite continuity (typed + PTT + blind recall) after auth seed; --suite all for RC. Evidence under .harness/agent-continuity-gauntlet/*/manifest.json with matching git SHA.
Anti-flake: clear owner/kernel surface before probes; per-run nonces; hard-fail on blind-recall / structural snapshot only; zero automatic retries on model wrongness.
Stress: offline JSONL + forbidden terminal reasons remain the default gate; live bridge probes stay optional until continuity terminal_reasons exist in the taxonomy.

Live gauntlet vs hermetic INV-6 coverage

Do not confuse these gates — a green live suite does not prove write-path contract rules, and hermetic unit tests do not prove bridge/LLM continuity.

Gate	What it covers	What it does not cover
Hermetic (`agent-logic-harness.sh` Swift filter: `KernelTurnRecordedProjectionTests`, `ChatTimelineContinuityTests`, `FloatingControlBarStateTests`, `RuntimeOwnerIdentityTests`)	stage/promote same key → one message pair; floating snapshot aliases main; structured agent identity; open-by-id hydrate preference; floating viewport derive / SoT; resources on producing message; agent preview = prompt; owner-swap preserves Firebase tokens; forbidden dual-write tripwires	Live bridge auth, LLM tool use, PTT hub, race/busy policy under a real runtime
Gauntlet `--self-check`	Bridge action registration (incl. R3 `ask_main_chat_no_wait` / `main_chat_busy_state`), resilience suite wiring, hermetic contract test presence in harness filter	Any live turn
Live `--suite continuity` / `agents` / `owner` / `prompts`	Typed + PTT + blind recall, spawn/status, owner swap probe, prompt regressions on a named bundle	stage/promote single-writer, snapshot alias, hydrate preference, viewport SoT (those stay hermetic)
Live `--suite resilience` (R1–R4)	Cold bridge launch, warm reuse, bridge busy/race rejection (R3; requires real `is_sending`/`is_streaming` once, latch only extends the race window), subagent launch+status (R4)	INV-6 write-path unit invariants above

--self-check fails if R3 race actions or the hermetic INV-6 test methods / harness filter classes drift away.

Verifying UI Changes (agent-swift)

Verify Swift UI changes programmatically with agent-swift (Accessibility-API CLI): setup, command reference, and key rules in docs/agent-swift-ui-verification.md. Never automate prod bundles.

Changelog Entries

After completing a desktop task with user-visible impact, add one fragment file under desktop/macos/changelog/unreleased/:

Example desktop/macos/changelog/unreleased/20260628-short-description.json:

{
  "change": "Your user-facing change description"
}

Guidelines:

Write from the user's perspective: "Fixed X", "Added Y", "Improved Z"
One sentence, no period at the end
Use a unique kebab-case filename so parallel PRs do not conflict
Tests, generated Swift, e2e harness files, and listed release-infra paths are already exempt
Internal-only production edits (dead-code deletion, refactors in Sources/) need an in-repo marker, not the no-changelog-needed PR label — that label is invisible after merge and reddens main. Add {"kind": "none"} under desktop/macos/changelog/unreleased/ instead
HTML is allowed for links: <a href='...'>text</a>
Do not edit CHANGELOG.json by hand; release automation regenerates it
Commit the fragment with your other changes (same commit is fine)

User Task Completion Reporting

When completing a task that was triggered by an app user request (bug report, feature request, support inquiry, etc.) and you have the user's email address, send them an email about the results using the omi-email skill:

node ../omi-analytics/scripts/send-email.js \
  --to "<user-email>" \
  --subject "<brief result summary>" \
  --body "<what was done, what they should expect, any next steps>"

Write as Matt (first person "I", not "we") — the user already has an ongoing email thread with us, so treat this as a casual continuation of that conversation, not a fresh introduction
Be concise and direct — they know the context, just share what was done and any next steps (e.g. "update the app")
Only send when there are meaningful results to share (don't email for internal-only changes)

FilesExpand file tree

AGENTS.md

Latest commit

History