Skip to content

Latest commit

 

History

History
261 lines (200 loc) · 15.3 KB

File metadata and controls

261 lines (200 loc) · 15.3 KB

Analytics

What Context for Claude reports about itself, where it lands, and how to ask it a question.

Before this existed the app was completely unmeasured. The only signals were GitHub release download counts — whose -latest pointer resets on every release, so the history was being destroyed weekly — and Cloud Run request logs, which see nothing from an install that never signs in. "How many people use this" could not be answered.

Where it goes

PostHog project 302298, the same project as the Omi macOS app, via https://us.i.posthog.com/batch/. The project token in AnalyticsSink.projectToken is public by design: it can only write events into that project, it reads nothing, and it already ships inside the Omi desktop binary.

Two products, one project, kept apart by two independent mechanisms:

  1. Every event name is prefixed cfc_.
  2. Every event carries app = "context-for-claude".
  3. No event sets $os_name. Omi's macOS retention, activation and weekly-actives queries all scope on properties.$os_name = 'macOS'. Setting it here would silently enrol every Context for Claude install into a four-month Omi trend line. macos_version carries the same information under a name nothing else queries.

Either of the first two alone is enough to separate the products; both are present so that whichever one a future query author reaches for, it works. The third is pinned by AnalyticsPayloadTests.testPayloadNeverSetsDollarOSName.

What is sent

The complete list is AnalyticsEvent — a closed enum, deliberately, so this file cannot go stale without a compile error somewhere. Every associated value is a number, a bool, or another closed enum; AnalyticsEventTests.testNoEventCarriesFreeFormText fails if a free string appears.

Event Answers
cfc_first_launch installs — the denominator
cfc_app_launched the only signal from someone who opens the app and does nothing
cfc_daily_active DAU, retention, and "how much" — see below
cfc_permission the setup funnel, per capability, snapshotted every launch
cfc_onboarding_step / cfc_onboarding_finished where first run is abandoned
cfc_account_state whether an Omi account is attached (never which)
cfc_capture_state mic / system audio / screen going live and stopping
cfc_gesture_fired ⌘ + ⌘ actually firing — inert without Accessibility
cfc_surface_opened which surfaces people open by hand, and by which route
cfc_surface_closed how long the timeline is actually read, bucketed
cfc_claude_handoff the user handing a question to Claude — the inverse of tool_*
cfc_control_used the twelve controls that answer a product question
cfc_tutorial_step the eleven-beat tutorial, where the product is actually taught
cfc_search_ran local search use, bucketed result counts only
cfc_first_artifact activation — the first time this install ever stored anything
cfc_update_outcome update health — a fleet that stops updating looks like a fleet nobody uses
cfc_fallback fail-open paths, mirroring ContextTelemetry.recordFallback

via is why cfc_surface_opened is worth reading

Four routes reach the search panel and four reach the timeline. Without a source they were one number, and the route is the product decision — a chord nobody uses and a menu row everybody uses were indistinguishable. It is emitted inside the three presenters rather than at each caller, because a per-caller emit measures whichever callers somebody remembered to touch. The bring-forward branch emits too, so summoning a panel that is already up is counted; before this it returned early and "brought to the front" was invisible.

Surface.rewind exists as of this change. Before it, openTheTimeline() reported .activity while opening RewindWindow — so the series named for the primary window was counting the demoted one, through one of its four routes, and the primary window reported as .search. Anything read from activity or search before 2026-08-21 describes the other surface.

cfc_surface_closed is bucketed, and that is the disclosure boundary

Dwell is the one question cfc_surface_opened structurally cannot answer: a timeline opened and abandoned in four seconds and one read for half an hour are one event each. A raw interval would be a per-person record of how long somebody sat with their own recorded screen, so the payload carries a DurationBucket and never a number of seconds.

The state machine is DwellClock, extracted rather than left inline because its failure is invisible in a window test. The timeline is .titled and .closable, so the X and ⌘W call AppKit's own close() and never reach dismiss(). A stash cleared only in dismiss() keeps running while the window is shut, and because re-opening takes the bring-forward branch — which deliberately does not restart the clock, since raising a window that is already up is the same visit — the next close reports one bucket spanning every minute in between. Both paths funnel through one reporter, and testAVisitAfterACloseMeasuresOnlyTheSecondVisit fails against an implementation that forgets to clear.

conversations on the rollup, because the minutes could not be a count

cfc_first_artifact fires once per install ever, and capture_minutes reads the same on the hundredth day as on the first — so "how much does this person actually produce in a day" had no answer. Counted at the three seams Engine closes a session through (the rollover inside append, the explicit close on pause and quit, and the sweep that closes what a crash left open), never for a close that threw, and cleared by UsageClock.reset() on the same day boundary as the minutes.

Deliberately not the account_rows cache: that is a copy of what the account already holds, and counting it would report another device's work as this Mac's.

What active_hours used to measure

UsageClock.noteActivity had no production caller — only its own definition and one test. The hour set was therefore filled only as a side effect of capture transitions inside mark(), which measures "hours in which capture toggled", not the "distinct hours in which anything at all happened" this document has always claimed. The documented reading — twelve active hours and twelve capture minutes is a person at a desk — was unreachable. ContextAnalytics.record now marks the hour for every event. Any active_hours read before 2026-08-21 is a capture-transition count.

What cfc_search_ran used to count

Keystrokes. It was emitted from SearchResultsModel.reload(), which runs on every keystroke with no debounce, once per panel open with an empty query via onAppear, and once per filter click — so typing a seven-letter word reported seven searches, and a panel nobody typed into reported one. It now fires when a question is committed. This is also why 82% of historical results bucket as many and none as zero: an empty query returns the newest captures.

cfc_first_artifact is the activation moment

Once per install, the first time a write actually lands, carrying kindconversation (a transcript segment) or screen (a frame). Emitted from EngineStore's two write paths, threaded through the write so a failed insert cannot report one: an attempt is not an artifact, and EngineStore catches and logs every failed insert.

Nothing else answers "did this install ever do the thing": cfc_capture_state reports a microphone being switched on, and cfc_daily_active's capture minutes look the same on the hundredth day as on the first. An install that captured all day and one that captured nothing were indistinguishable.

There is no memory kind, and that is a fact about the product. This app never stores a memory: create_memory writes to the Omi account from context-for-claude-mcp, a separate short-lived process that cannot report anything (see below), and the local account_rows table is a copy of what the account already holds — first-storing a downloaded memory would fire this for an install that has captured nothing at all.

step_total describes the run, not the enum

The emit used to pass all seven OnboardingStep cases, but a signed-in user's itinerary is six and their events skip the sign-in index — which put a permanent, real-looking cliff in the funnel that was a reinstall arriving already known rather than a person giving up. It now reports the length of the itinerary the run is actually on, so the value is 6 or 7 and the funnel is readable. Both values are reported, so cross-version comparison still works.

cfc_onboarding_finished is once per install, across a relaunch

Granting Screen Recording only takes effect in a new process, so onboarding restarts the app from its own middle and the run that reaches the last card often had no step transition of its own. The start instant is therefore persisted (context.analytics.onboardingStartedAt) rather than held in memory, and the completion is spent against a second default (context.analytics.onboardingFinishedReported) so that Settings' "Run setup again" cannot add a second install-shaped completion. Before that, four of the five reporting installs had permissions granted and exactly one had ever sent the event, which made the setup funnel unreadable.

cfc_daily_active is the spine

One event per install per local calendar day, carrying the day's rollup:

  • tool_calls_total, tool_calls_distinct, and one tool_<name> per MCP tool
  • capture_minutes, screen_minutes — wall clock, not a count of start/stop events
  • active_hours — distinct hours in which anything happened. Twelve active hours and twelve capture minutes is a person at a desk; one active hour and 600 capture minutes is a laptop left open.
  • signed_in, airgapped, idle

DAU is this event. Retention is this event, cohorted. "How much do they use it" is tool_calls_total — the only evidence that captured context is reaching a model rather than accumulating on a disk.

What is never sent

No transcript, no OCR text, no window or app names, no URLs, no file paths, no search queries, no MCP tool arguments, no email, no account id, no screenshots. A reader of AnalyticsEvent.swift can see the entire disclosure — that is the property the closed enum exists to guarantee.

Identity

distinct_id is cfc_ + 16 hex characters of SHA256("context-for-claude/analytics/v1" + installId).

The install id is the same UUID ClientDevice uses, under a different salt. That separation is the whole privacy argument: the backend's X-Device-Id-Hash keys the user's own captured rows and is joinable to their account, so an equal id would make every "anonymous" event trivially re-identifiable by anyone holding both datasets. Salted apart, the two cannot be linked without the original UUID, which never leaves the Mac.

The id does not change on sign-in. Knowing that an install has an account answers every product question here; knowing which would turn an anonymous series into a per-person record of when somebody's microphone was on.

Changing AnalyticsIdentity.salt re-anonymises every install and restarts every retention curve. Don't.

The three refusals

  1. Airgap Mode drops events, it does not defer them. Every other NetworkEgress.Client queues its work and sends when the switch goes off. An analytics event doing that would mean Airgap Mode delayed the disclosure rather than preventing it. Those days are simply not measured.

    Note that the Settings switch for Airgap Mode no longer exists — it was removed in 1.0.9 and ExclusionEngine.setAirgapMode has had no caller since. The flag is still reachable two ways, and both are why the guard stays: an exclusions.json written before 1.0.9 (or by hand) still carries it, and ExclusionSet.make forces it on whenever the exclusion configuration fails closed — a config we cannot parse may carry exclusions we cannot express. The second case is the one that matters in practice: it means a machine with a corrupt config stops reporting rather than reporting from a state where it cannot honour the user's exclusions.

  2. Only the shipping app reports. ContextPaths.isShippingBundle gates the whole path. This is not tidiness: for this app's first three weeks the Cloud Run logs show a Context for Claude/1 user agent from up to twenty machines a day — the team's own builds, indistinguishable in aggregate from users.

    The gate asks "is this the release?", not "is this not a dev build?" — it used to ask the second, via ContextPaths.isDevelopmentBuild, which is derived from an identifier that falls back to the shipping one for any process that is not ours. swift test runs under com.apple.dt.xctest.tool and so counted as production: the suite POSTed to this project from the real spool for as long as it has existed, 92 events before it was noticed, which is all of cfc_gesture_fired and two thirds of cfc_search_ran. Anything queried before 2026-08-19 carries that noise.

  3. Nothing user-authored, ever — enforced by construction, not by review.

MCP tool calls cross a process boundary

context-for-claude-mcp is spawned by Claude over stdio, several at a time, and killed without warning. It cannot report anything itself: a short-lived process that POSTs on exit either blocks Claude's shutdown or loses the event.

So the MCP process counts and the app reports. ToolCallLedger (in ContextCore) is the seam — a multi-writer counter file under an flock, drained destructively by the app's daily rollup. It is separate from QueryStamp because that file is monotonic-latest ("did Claude just call us?") and this one is cumulative ("how much?"); one file cannot be both.

Both hold a tool name and nothing else.

Delivery

Batched, not per-event. One request per event would be absurd at ~10 events a day and, worse, would make the app's network fingerprint track the user's activity in real time — a request the moment a recording starts, another the moment a search runs. Batching on a 60-second timer decouples when we send from what the person just did. That is a privacy property, not only an efficiency one.

The spool is durable (survives relaunch), capped at 500 events dropping oldest-first, and drains 50 per request. A failed send keeps the events; a 4xx other than 429 drops the batch, because keeping an unsendable batch would park every later event behind it forever.

Verifying it end to end

A release build cannot be run under a debugger on the machine that wrote it, and a sink nobody has watched deliver is a sink that has never worked. So:

CONTEXT_ANALYTICS_FORCE=1 /path/to/Context\ for\ Claude.app/Contents/MacOS/Context\ for\ Claude

This overrides refusal 2 only. Events sent this way are indistinguishable from real ones and land in production series — use a throwaway session, not a day of ordinary work.

Then query PostHog:

SELECT event, count() AS n, count(DISTINCT person_id) AS installs
FROM events
WHERE timestamp > now() - INTERVAL 1 DAY AND properties.app = 'context-for-claude'
GROUP BY event ORDER BY n DESC

Asking it questions

~/.claude/skills/omi-analytics/scripts/ph.py --preset cfc covers actives, installs, tool-call volume and the permission funnel. Anything else is HogQL against properties.app = 'context-for-claude'.