Max Litruv Boonzaayer
e058165830
Bump Vite 8, matrix-js-sdk, and related deps; fix immer/vanilla-extract imports for new majors. Replace blocking sync error dialog with background recovery on wake/offline, and disable Vite auto-open in dev. Add feature handoff documentation index.
98 lines
3.4 KiB
Markdown
98 lines
3.4 KiB
Markdown
# Voice & video calls — handoff
|
|
|
|
## Summary
|
|
|
|
Paarrot supports Matrix RTC voice/video calls via MSC3401 (`org.matrix.msc3401.call.member`) and LiveKit. Calls can be docked in the sidebar, support screen share, remote cursors, mute/deafen, and are wired to the local HTTP API for external control.
|
|
|
|
## User-facing behavior
|
|
|
|
- Join/start calls from room UI (when homeserver exposes RTC/LiveKit via `.well-known`)
|
|
- Docked call panel in sidebar; full overlay mode
|
|
- Mute, deafen, camera, screen share controls
|
|
- Call participants shown in room nav (`CallParticipantsIndicator`)
|
|
|
|
## Architecture
|
|
|
|
```
|
|
useCallService (CallProvider)
|
|
→ CallService (LiveKit room, Matrix call member state)
|
|
→ useRtcConfig: .well-known m.rtc + LiveKit focus
|
|
→ UI: CallOverlay, DockedCallPanel, SidebarDockedCallPanel
|
|
→ getCallService() global ref for Electron API (paarrot-api.ts)
|
|
```
|
|
|
|
## Key files
|
|
|
|
| Path | Role |
|
|
|------|------|
|
|
| `cinny/src/app/features/call/useCall.ts` | React context, global CallService ref |
|
|
| `cinny/src/app/features/call/CallService.ts` | Core call logic |
|
|
| `cinny/src/app/features/call/types.ts` | CallState, ActiveCall, events |
|
|
| `cinny/src/app/features/call/useRtcConfig.ts` | Homeserver RTC / LiveKit discovery |
|
|
| `cinny/src/app/features/call/CallOverlay.tsx` | Full-screen call UI |
|
|
| `cinny/src/app/features/call/DockedCallPanel.tsx` | Docked UI |
|
|
| `cinny/src/app/features/call/SidebarDockedCallPanel.tsx` | Sidebar variant |
|
|
| `cinny/src/app/features/call/useRemoteCursor.ts` | Screen-share cursor sync |
|
|
| `cinny/src/app/features/call/CallSounds.ts` | Join/leave sounds |
|
|
| `cinny/src/app/features/room-nav/CallParticipantsIndicator.tsx` | Nav badge |
|
|
| `cinny/src/types/matrix/room.ts` | `StateEvent.CallMember` |
|
|
|
|
## Data model
|
|
|
|
| Key | Scope | Notes |
|
|
|-----|-------|-------|
|
|
| `org.matrix.msc3401.call.member` | Room state | Call membership |
|
|
| LiveKit JWT / room | Ephemeral | From homeserver RTC focus |
|
|
|
|
Call mute/deafen/video state lives in `CallService` memory, not Matrix account data.
|
|
|
|
## Dependencies
|
|
|
|
- `livekit-client` (verify in package.json)
|
|
- Homeserver `.well-known` RTC configuration
|
|
- Browser/Electron media permissions (mic, camera, screen capture)
|
|
- Electron `desktopCapturer` for screen share (main process)
|
|
|
|
## Integration points
|
|
|
|
- **local-api**: mute/deafen/status endpoints
|
|
- **Stream Deck**: uses call mute/deafen via API
|
|
- **Room nav**: participant indicators
|
|
- **Electron main**: screen capture IPC if applicable
|
|
|
|
## Testing
|
|
|
|
### Manual
|
|
|
|
1. Use a homeserver with Matrix RTC + LiveKit configured.
|
|
2. Start voice call in a room; verify audio both ways.
|
|
3. Toggle mute/deafen; verify API `/status` reflects state.
|
|
4. Screen share; verify remote cursor if enabled.
|
|
5. Dock/undock call panel.
|
|
|
|
### Automated
|
|
|
|
- None identified
|
|
|
|
## Known issues & gotchas
|
|
|
|
- `callSupported` false until `.well-known` RTC fetch completes — UI should hide call buttons when unsupported
|
|
- `getCallService()` must be set before API mute actions work
|
|
- Platform differences: screen capture APIs differ Electron vs browser vs Android
|
|
|
|
## Future work
|
|
|
|
- Call recording, noise suppression settings
|
|
- Better reconnection / call transfer
|
|
- MSC3401 spec tracking as it stabilizes
|
|
|
|
## Related docs
|
|
|
|
- [local-api HANDOFF](../local-api/HANDOFF.md)
|
|
- Matrix MSC3401 (upstream spec)
|
|
|
|
## Add your extra things here
|
|
|
|
- `fetchWellKnownWithRTC`, `getLiveKitFocus`, `getLiveKitHomeserverPriority` in `useRtcConfig.ts`
|
|
- Global singleton: `globalCallServiceInstance` in `useCall.ts` for non-React consumers
|