forked from BasedHardware/omi
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathphone-capture.yaml
More file actions
131 lines (121 loc) · 8.91 KB
/
Copy pathphone-capture.yaml
File metadata and controls
131 lines (121 loc) · 8.91 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
# E2E Flow: Phone Mic Capture — record audio via phone microphone, verify conversation is created
# Tests: Start phone mic recording, capture live transcription, process conversation, verify in conversations list
# Core feature: recording conversations using the phone's built-in microphone (no Omi device required)
version: 2
name: phone-capture
description: Phone mic capture flow — start recording via central mic button, capture ~3 minutes of real audio, view live transcription, process conversation, verify it appears in conversations list
app: com.friend.ios.dev
evidence:
video: true
covers:
- app/lib/widgets/bottom_nav_bar.dart
- app/lib/providers/capture_provider.dart
- app/lib/pages/conversation_capturing/page.dart
- app/lib/pages/conversations/widgets/processing_capture.dart
preconditions:
- auth_ready # User signed in and onboarding completed. App shows home screen.
- no_omi_device # No Omi BLE device paired or connected. The central mic button must be visible in the bottom nav bar. If an Omi device is paired, disconnect it via Settings > Device > Disconnect Device first.
- microphone_permission # Android: `adb shell pm grant com.friend.ios.dev android.permission.RECORD_AUDIO`. iOS: tap Allow on permission dialog.
steps:
- id: S1
name: Verify home screen with mic button visible
do: "Verify the home screen shows: Omi icon (top-left), settings icon (top-right), 'Conversations' section header with filter tabs (All, Starred, Work, Personal), and importantly — a large purple circular mic button (80x80) in the center of the bottom navigation bar. The bottom nav has 4 tab icons (Home, Action Items, Memories, Apps) with the mic button centered between them."
verify: true
expect:
- kind: text_visible
values: ["Conversations"]
- kind: interactive_count
min: 5
evidence:
- screenshot: step-S1.webp
note: "The central mic button only appears when NO Omi BLE device is connected (isOmiDeviceConnected == false). The button is a GestureDetector with a purple circle (Colors.deepPurple) and white microphone icon (FontAwesomeIcons.microphone). It is positioned via Positioned widget at bottom: 40, centered horizontally. ADB coordinates: approximately (540, 2280) on 1080x2400 device."
- id: S2
name: Tap mic button to start recording
do: "Tap the central purple mic button to start phone mic recording. The button changes to red with a stop icon while initializing. If this is the first time, a microphone permission dialog may appear — grant it. ADB tap at (540, 2280) on 1080x2400 device."
verify: true
expect:
- kind: text_visible
values: ["Listening"]
- kind: interactive_count
min: 2
evidence:
- screenshot: step-S2.webp
note: "Tapping the mic button calls captureProvider.streamRecording() which: (1) sets state to RecordingState.initialising, (2) requests mic permission, (3) sends geolocation, (4) configures PCM16 codec at 16kHz sample rate, (5) starts mic streaming audio bytes to backend via WebSocket, (6) navigates to ConversationCapturingPage. The app bar shows '🎙️ Listening'."
- id: S3
name: Verify capturing page — waiting for transcripts
do: "Verify the ConversationCapturingPage shows: back arrow (top-left), '🎙️' emoji + 'Listening' title in app bar, and the message 'Waiting for transcripts or photos...' in the center. No 'Process Now' button visible yet (it only appears after segments/photos arrive). Two tabs exist: Transcripts (tab 0, active) and Summary (tab 1)."
verify: true
expect:
- kind: interactive_count
min: 1
evidence:
- screenshot: step-S3.webp
note: "The ConversationCapturingPage has a TabBarView with 2 tabs. Tab 0 shows transcripts/photos, tab 1 shows silence timeout info and 'No summary yet'. The floating action buttons (Process Now + Mute) only appear when provider.segments.isNotEmpty || provider.photos.isNotEmpty."
- id: S4
name: Speak and verify live transcription appears
do: "Speak clearly into the phone microphone for at least 30 seconds. Verify that transcript segments begin appearing on the Transcripts tab. Each segment shows speaker label (e.g., 'Speaker 0') and the transcribed text. The 'Process Now' button (yellow/amber with stop icon) and Mute button (grey circle with mic_off icon) should now appear at the bottom."
verify: true
expect:
- kind: text_visible
values: ["Process Now"]
- kind: interactive_count
min: 3
evidence:
- screenshot: step-S4.webp
note: "Transcript segments arrive via WebSocket as TranscriptSegmentEvent messages. Each segment has speakerId, text, confidence, startTime, endTime. The UI auto-scrolls to show new segments. Speaker labels can be tapped to open NameSpeakerBottomSheet for renaming. The 'Process Now' button is Color(0xFFFFB800) with FontAwesomeIcons.stop icon."
- id: S5
name: Continue recording for 3 minutes
do: "Continue speaking or playing audio near the phone microphone for approximately 3 minutes total. The transcript should grow with multiple segments. Optionally test the Mute button: tap the grey mic_off circle to pause recording (button turns red, status changes to '🔇 Muted'), then tap again to resume (button turns grey, status returns to '🎙️ Listening')."
verify: true
expect:
- kind: interactive_count
min: 3
evidence:
- screenshot: step-S5.webp
note: "Mute/unmute calls provider.stopStreamRecording() and provider.streamRecording() respectively. While muted, the mic stops sending bytes to the WebSocket but the conversation remains in-progress on the backend. The app bar emoji changes between 🎙️ (recording) and 🔇 (muted). Recording duration is not shown on-screen but can be calculated from segment timestamps."
- id: S6
name: Tap Process Now to end recording
do: "After ~3 minutes of recording, tap the 'Process Now' button (yellow/amber, bottom-center). If 'show summarize confirmation' preference is enabled, a confirmation dialog appears with title 'Finished?' asking 'Stop Recording?', with a 'Don't ask again' checkbox and Cancel/Confirm buttons. Tap 'Confirm' (or if preference is disabled, recording stops immediately)."
verify: true
expect:
- kind: text_visible
values: ["Conversations"]
- kind: interactive_count
min: 4
evidence:
- screenshot: step-S6.webp
note: "Process Now calls _stopConversation which: (1) stops phone mic recording via provider.stopStreamRecording(), (2) calls provider.forceProcessingCurrentConversation() which POSTs to /v1/conversations, (3) navigates back to home. The backend processes the conversation (transcription, diarization, summarization) and returns a ConversationEvent. The new conversation is added to the conversations list with isNew=true."
- id: S7
name: Verify new conversation appears in conversations list
do: "On the home screen, verify a new conversation has appeared at the top of the Conversations list (under today's date). The conversation card shows the title (auto-generated from transcript content), timestamp, and duration. The conversation was just created from the phone mic recording."
verify: true
expect:
- kind: text_visible
values: ["Conversations"]
- kind: interactive_count
min: 5
evidence:
- screenshot: step-S7.webp
note: "The new conversation appears via conversationProvider.upsertConversation(). It may briefly show 'Processing...' status before the full conversation with summary is available. The conversation source is ConversationSource.phone (not omi or desktop). Conversations are sorted by date with newest first."
- id: S8
name: Open new conversation detail
do: "Tap the newly created conversation card to open the conversation detail page. Verify the detail page shows: back arrow, conversation title, transcript tab with speaker-labeled segments matching what was recorded, and overview/action items tabs."
verify: true
expect:
- kind: interactive_count
min: 4
evidence:
- screenshot: step-S8.webp
note: "The conversation detail page (ConversationDetailPage) shows 3 tabs: Overview (summary + action items), Transcript (full speaker-diarized transcript), and Moments (key moments extracted by AI). The transcript should contain the audio recorded during the ~3 minute session. Each speaker segment shows speaker name and transcribed text."
- id: S9
name: Return to home
do: "Press the back arrow or Android back button to return to the home screen. Verify the conversations list is visible with the new conversation still at the top."
verify: true
expect:
- kind: text_visible
values: ["Conversations"]
- kind: interactive_count
min: 4
evidence:
- screenshot: step-S9.webp
note: "KEYCODE_BACK from conversation detail returns to home safely (unlike Action Items page). The conversation persists in the list and can be revisited."