| title | Backend Setup |
|---|---|
| icon | server |
| description | Set up the Omi backend infrastructure for local development. This guide covers Google Cloud, Firebase, OAuth, and all required services. |
The Omi backend powers all AI capabilities including transcription, conversation processing, chat, and integrations. This guide will help you set up a local development environment.
flowchart LR
subgraph External["External Services"]
OAI[OpenAI]
STT[Parakeet / Modulate]
Pine[Pinecone]
Redis[Redis]
end
subgraph Google["Google Cloud"]
FB[Firebase]
GCS[Cloud Storage]
end
subgraph Backend["Your Backend"]
API[FastAPI Server]
Push[Pusher Service]
end
subgraph App["Omi App"]
Flutter[Flutter Client]
end
Flutter <-->|Ngrok| API
API --> OAI
API --> DG
API --> Pine
API --> Redis
API --> FB
API --> GCS
API --> Push
<iframe width="560" height="315" src="https://www.youtube.com/embed/JyXQM0B0Gnc?si=_2W2GU5Qr7dn7gC7" title="Omi Backend Setup Video Guide" frameBorder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen ></iframe>
Before starting, gather these API keys and credentials:
| Service | Purpose | Get Key | |---------|---------|---------| | OpenAI | AI language models | [Get Key](https://platform.openai.com/settings/organization/api-keys) | | Modulate | Speech-to-text fallback | Provision `MODULATE_API_KEY` | | Parakeet | Self-hosted speech-to-text | Configure `HOSTED_PARAKEET_API_URL` | | Redis (Upstash) | Caching & sessions | [Get Key](https://console.upstash.com/) | | Pinecone | Vector database | [Get Key](https://app.pinecone.io/organizations/-/projects/-/api-keys) | | Hugging Face | Voice detection | [Get Key](https://huggingface.co/settings/tokens) | | Service | Purpose | Get Key | |---------|---------|---------| | Modal | Serverless deployment | [Get Key](https://modal.com/settings#tokens) | | GitHub | Firmware updates | [Get Key](https://github.com/settings/tokens) | | Google Maps | Location features | [Get Key](https://console.cloud.google.com/google/maps-apis/credentials) | | Typesense | Search functionality | [Get Key](https://cloud.typesense.org/clusters) | | Stripe | Payment processing | [Get Key](https://dashboard.stripe.com/apikeys) | New to backend development? Install [Homebrew](https://docs.brew.sh/Installation) (macOS/Linux) or [Chocolatey](https://chocolatey.org/install) (Windows) first - they make installing tools much easier.```bash brew install google-cloud-sdk ``` ```powershell choco install gcloudsdk ``` The SDK is usually pre-installed in Nix environments. Otherwise, follow the [official installation guide](https://cloud.google.com/sdk/docs/install). Go to the [Google Cloud Console](https://console.cloud.google.com/) and enable these APIs:
<CardGroup cols={2}>
<Card title="Cloud Resource Manager API" icon="sitemap" href="https://console.cloud.google.com/apis/library/cloudresourcemanager.googleapis.com">
Required for project management
</Card>
<Card title="Firebase Management API" icon="fire" href="https://console.cloud.google.com/apis/library/firebase.googleapis.com">
Required for Firebase integration
</Card>
</CardGroup>
```bash
gcloud auth login
gcloud config set project <project-id>
gcloud auth application-default login --project <project-id>
```
This generates credentials at `~/.config/gcloud/application_default_credentials.json`.
OAuth is required for user authentication. You need to configure both Google and Apple sign-in.
1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials) 2. Click **Create Credentials** → **OAuth 2.0 Client ID** 3. Select **Web application** as the application type 4. Set a name (e.g., "Omi Backend Auth") Under **Authorized JavaScript origins**, add: - `https://your-domain.com` (production) - `https://your-ngrok-domain.ngrok-free.app` (local development) Under **Authorized redirect URIs**, add: - `https://your-domain.com/v1/auth/callback/google` - `https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/google` Click **Create** and save your **Client ID** and **Client Secret** for the `.env` file. Go to **APIs & Services → OAuth consent screen**: - Fill in required app information - Add your domain to **Authorized domains** - Add scopes: `openid`, `email`, `profile` Apple OAuth requires a paid Apple Developer account ($99/year).<Steps>
<Step title="Create App ID">
1. Go to [Apple Developer Console → Identifiers](https://developer.apple.com/account/resources/identifiers/list)
2. Create a new App ID with **Sign In with Apple** capability enabled
3. Note your Bundle ID
</Step>
<Step title="Create Services ID">
1. Create a new **Services ID** (this becomes your `APPLE_CLIENT_ID`)
2. Configure **Sign In with Apple** for this Services ID
3. Add authorized domains and return URLs:
- `https://your-domain.com/v1/auth/callback/apple`
- `https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/apple`
</Step>
<Step title="Create Private Key">
1. Go to **Keys** in Apple Developer Console
2. Create a new key with **Sign In with Apple** enabled
3. Download the `.p8` file and note the **Key ID**
4. Note your **Team ID** from your Apple Developer account
</Step>
<Step title="Configure Firebase">
1. Go to [Firebase Console](https://console.firebase.google.com/) → Authentication → Sign-in method
2. Enable **Apple** and add your configuration:
- Client ID: Your Services ID
- Team ID: Your Apple Developer Team ID
- Key ID: From your private key
- Private Key: Contents of your .p8 file
</Step>
</Steps>
<Note>
Your Apple environment variables will be:
- `APPLE_CLIENT_ID`: Your Services ID
- `APPLE_TEAM_ID`: Your Apple Developer Team ID
- `APPLE_KEY_ID`: The Key ID from step 3
- `APPLE_PRIVATE_KEY`: The full contents of your .p8 file (including BEGIN/END lines)
</Note>
```bash git clone https://github.com/BasedHardware/Omi.git cd Omi/backend ``` The backend Docker images run Python 3.11, so use Python 3.11 locally as well. If your package manager installs a newer default Python, install or select Python 3.11 before creating the virtual environment.
<Tabs>
<Tab title="macOS">
```bash
brew install python@3.11 git ffmpeg opus uv
uv --version
```
</Tab>
<Tab title="Windows">
Install Python 3.11 from [python.org](https://www.python.org/downloads/windows/), then install the remaining system tools and `uv`:
```powershell
choco install git.install ffmpeg
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
python --version # should print 3.11.x
uv --version
```
<Note>
The Python `opuslib` package still needs a native `libopus` DLL on `PATH`. One common setup is MSYS2 UCRT64:
install MSYS2, run `pacman -S mingw-w64-ucrt-x86_64-opus`, add `C:\msys64\ucrt64\bin` to `PATH`,
then open a new shell and verify with `where.exe opus.dll`.
</Note>
</Tab>
<Tab title="Linux/Nix">
Python, Git, and FFmpeg are typically pre-installed. Install opus and `uv` via your package manager if needed, or use the standalone installer:
```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
uv --version
```
</Tab>
</Tabs>
<Tabs>
<Tab title="macOS/Linux">
```bash
# From backend/
./scripts/sync-python-deps.sh
source .venv/bin/activate
```
</Tab>
<Tab title="Windows PowerShell">
```powershell
# From backend/
bash ./scripts/sync-python-deps.sh
.venv\Scripts\Activate.ps1
```
</Tab>
</Tabs>
You should see `(.venv)` at the beginning of your command prompt.
```bash
./scripts/update-python-lock.sh
```
The refresh preserves already-locked package versions by default. Use `PYLOCK_UPGRADE=1 ./scripts/update-python-lock.sh` only when intentionally upgrading dependency versions.
Edit `.env` and fill in your API keys (see [Environment Variables](#environment-variables) below).
Skip this if you don't need webhook functionality.
```bash
cd pusher
cp .env.template .env
```
Edit the `.env` file and set `SERVICE_ACCOUNT_JSON` to your Google credentials string.
Start the service:
```bash
cd ..
uvicorn pusher.main:app --reload --env-file .env --port 8000
```
Optionally expose via Ngrok for external access.
1. Create an account on [Typesense](https://typesense.org/)
2. Create a collection named `conversations` using the schema in `typesense/conversations.schema`
3. Install the [Firebase Typesense extension](https://console.firebase.google.com/project/_/extensions/install?ref=typesense/firestore-typesense-search@2.0.0-rc.1)
Configuration for the extension:
| Setting | Value |
|---------|-------|
| Firestore Collection Path | `users/{userId}/conversations` |
| Firestore Collection Fields | `structured,created_at,discarded,started_at,id,finished_at,geolocation,userId` |
<Note>
If you have existing data, create a `typesense_sync` collection and add a document named `backfill` with `{'trigger': true}`.
</Note>
Ngrok exposes your local backend to the internet so the Omi app can connect.
1. Sign up at [ngrok.com](https://ngrok.com/) and install Ngrok
2. Authenticate with your account
3. Start the tunnel:
```bash
ngrok http --domain=your-domain.ngrok-free.app 8000
```
Note the public URL (e.g., `https://your-domain.ngrok-free.app`).
| Flag | Purpose |
|------|---------|
| `--reload` | Auto-restart on code changes |
| `--env-file .env` | Load environment variables |
| `--host 0.0.0.0` | Listen on all interfaces (optional) |
| `--port 8000` | Port to listen on |
```
API_BASE_URL=https://your-domain.ngrok-free.app/
```
<Warning>
Don't forget the trailing `/` in the URL!
</Warning>
Make sure your OAuth redirect URIs in Google Cloud Console and Apple Developer Console include your Ngrok URL.
```bash
deactivate
```
To resume later, just activate the virtual environment again.
Add this to `utils/stt/vad.py`:
```python
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
```
```bash
./scripts/sync-python-deps.sh
```
Complete reference for all .env variables:
We use black for code formatting with a line length of 120 characters.
source .venv/bin/activate
uv pip install blackTo auto-format on commit, install the pre-commit hook from the repository root:
ln -s -f ../../scripts/pre-commit .git/hooks/pre-commitSet up the Flutter app for development Understand the backend architecture Learn how the chat feature works How conversations and memories are stored
Report bugs or request features Get help from the community