Add bot-driven room app UI via im.paarrot.ui state.
All checks were successful
Trigger cinny-mobile / dispatch (push) Successful in 2s

Bots can publish a declarative panel UI that takes over a room; user
actions are sent as im.paarrot.ui.action timeline events.
This commit is contained in:
2026-08-09 13:16:43 +10:00
parent 32bf2cbed5
commit 8a68a1e30a
15 changed files with 866 additions and 5 deletions

View File

@@ -0,0 +1,93 @@
# Room App UI — handoff
## Summary
Bots can publish a declarative UI into Matrix room state (`im.paarrot.ui`). Paarrot takes over the room view with that UI. User interactions are sent as timeline events (`im.paarrot.ui.action`), not chat messages. The bot updates state to re-render panels.
## User-facing behavior
- When a room has a valid `im.paarrot.ui` state event, opening the room shows the app UI instead of the chat timeline (forum rooms still win over app UI).
- **Show chat** returns to the normal timeline without clearing bot state; a banner offers **Show app** to go back.
- Action events are hidden from the chat timeline (same as confetti relay events).
- Room settings → Permissions includes **Set Room App UI** and **Send Room App Actions**.
## Architecture
```
Bot → state im.paarrot.ui
→ Room.tsx detects via useStateEvent
→ RoomAppView → RoomAppSchema (declarative nodes)
→ user clicks → mx.sendEvent(im.paarrot.ui.action)
→ Bot updates im.paarrot.ui → live re-render
```
## Key files
| Path | Role |
|------|------|
| `cinny/src/types/matrix/room.ts` | `StateEvent.PaarrotUi`, `MessageEvent.PaarrotUiAction`, node types |
| `cinny/src/app/utils/room.ts` | `getPaarrotUiContent`, `hasRoomAppUi` |
| `cinny/src/app/features/room/Room.tsx` | Takeover branch + Show chat / Show app |
| `cinny/src/app/features/room-app/` | `RoomAppView`, schema renderer, CSS sanitize/scope |
| `cinny/src/app/features/room/RoomTimeline.tsx` | Hides `im.paarrot.ui.action` |
| `cinny/src/app/features/room-settings/permissions/usePermissionItems.ts` | Permission labels |
## Data model
| Event | Kind | Content |
|-------|------|---------|
| `im.paarrot.ui` | state (`""`) | `{ version, title?, css?, root }` |
| `im.paarrot.ui.action` | timeline | `{ action, component_id, value?, values?, ui_event_id? }` |
See [samples/](./samples/) for pasteable Developer Tools payloads.
MVP node types: `panel`, `row`, `text`, `button`, `input`, `select`, `image`, `spacer`.
## Dependencies
- folds UI primitives
- matrix-js-sdk `sendEvent` / state sync
- No bot-supplied JavaScript; CSS is sanitized and scoped under `[data-room-app]`
## Integration points
- Same room takeover pattern as forums (`Room.tsx` branch)
- Developer Tools can send state/events for demos without a bot process
## Testing
### Manual
1. In Developer Tools → Send State Event, type `im.paarrot.ui`, paste `samples/trivia-ui-state.json`.
2. Open the room — app UI should replace the timeline.
3. Click an answer — an `im.paarrot.ui.action` event is sent (hidden in timeline; visible in raw/dev tools).
4. Click **Show chat**, then **Show app**.
5. Clear state content / redact state to restore normal chat.
### Automated
- None yet
## Known issues & gotchas
- CSS scoping is best-effort, not a full CSS parser
- Images allow `mxc://`, `https://`, and `http://` only
- Forum rooms take priority over app UI
- `sendEvent(..., as any)` cast mirrors other custom event types
## Future work
- `form` / `tabs` / `markdown` / `progress` nodes
- Bot SDK helpers
- Stronger CSS sandbox (Shadow DOM)
- Encrypt custom action payloads helpers if needed
## Related docs
- [samples/trivia-ui-state.json](./samples/trivia-ui-state.json)
- [samples/ui-action-event.json](./samples/ui-action-event.json)
## Add your extra things here
- Power level: bots need PL ≥ `state_default` (usually 50) or an explicit level for `im.paarrot.ui`
- Action events use normal message event power levels unless overridden for `im.paarrot.ui.action`