Design for a poseable mannequin plus glTF/GLB import overlay in Umamo: world axes, orbit camera, draw path, UMA persistence, UI, risks. docs/seams.md lists the RenderDevice additions the feature needs: PrimitiveTopology, DepthState, depthAttachment, WorldToClip, ReferenceMeshSpec. sketch/ mirrors the algorithms in Kotlin. verify/verify.py proves them: 30 checks, all passing. Two findings: accessor byteOffset must be a multiple of the component size, and the screen-right basis collapses to a zero vector at pitch +/-90 degrees, so pitch is clamped to an open interval.
205 lines
10 KiB
Markdown
205 lines
10 KiB
Markdown
# 3D model reference layer
|
||
|
||
## What it is for
|
||
|
||
A drawing aid that sits in the viewport over the puppet. Five uses, in the order people
|
||
actually reach for them:
|
||
|
||
1. **Proportion check.** Is this head too big for this body. Umamo has no body, so today the
|
||
answer comes from memory or a photo pinned next to the window.
|
||
2. **Anatomy reference.** Where the elbow actually is in a pose, where the shoulder joint sits,
|
||
how a ribcage turns.
|
||
3. **Foreshortening.** A hand coming toward the camera. A 2D reference cannot show this; a
|
||
3D one can, and this is the one use that justifies the feature existing at all.
|
||
4. **Scale reference.** "This puppet is 160 cm tall." Everything else follows from that.
|
||
5. **Line-art underlay.** Pose the mannequin, lower its opacity, draw over it.
|
||
|
||
It is not a 3D modeler, and the distinction matters for every decision below.
|
||
|
||
## Non-goals
|
||
|
||
- No 3D puppet. The puppet stays a 2D deforming mesh set.
|
||
- No glTF export. `TODO.md` already lists glTF export as a far-future item; it is a different
|
||
problem and shares nothing with this except the word glTF.
|
||
- No lighting, no material system, no texture sampling on the reference layer. Flat fill,
|
||
wireframe, or vertex-normal shading. Three options, no more.
|
||
- No animation playback on the reference layer. Pose is a static state.
|
||
- No physics, no cloth, no bendy bones. `TODO.md` § Future Feature Wishes already wants a real
|
||
skeleton for the puppet; that is a separate project and this one must not become its
|
||
accidental first installment.
|
||
- Nothing from the reference layer reaches CMO3 or MOC3 export. It is editor-only. See § Risks.
|
||
|
||
## World-space convention
|
||
|
||
Umamo's world space is X right, Y depth, Z up. `PuppetModel.worldOriginX` and `worldOriginZ`
|
||
already exist, and the viewport already draws the X and Z axis lines through the world origin
|
||
(`module/render/src/jvmTest/.../gl/WorldAxisLinesTest.kt`).
|
||
|
||
The reference layer uses that same frame. No new axis convention, no conversion. A mannequin
|
||
standing at the world origin is already aligned with the drawing, and at yaw = pitch = 0 the
|
||
reference camera must agree exactly with `ViewportCamera.worldToNdc`. That agreement is the
|
||
anchor of the whole design: the reference layer at rest sits exactly in the drawing plane, and
|
||
rotating it is the only thing that separates it from the puppet.
|
||
|
||
Verified in `verify/verify.py`: at yaw 0 pitch 0 the camera basis is forward +Y, right +X,
|
||
up +Z, and a point one world unit right projects to NDC x = 2·zoom/viewportWidth, which is
|
||
`ViewportCamera.worldToNdc`'s value.
|
||
|
||
Units are model pixels: one world unit is one model pixel. A mannequin therefore needs a
|
||
scale stated in human units, not pixels. `referenceHeightCm` (default 160) plus the document's
|
||
puppet height in world units gives the scale, and that is what makes the proportion check
|
||
possible at all — a head is about 1/7.5 of standing height, and the mannequin encodes that as
|
||
data rather than as a guess.
|
||
|
||
## Data model
|
||
|
||
```
|
||
ReferenceLayer
|
||
source BuiltInMannequin(proportions) | ImportedGltf(assetKey, sceneIndex)
|
||
transform position (x, y, z), scale, rotation (yaw, pitch, roll)
|
||
pose Map<JointId, Euler(x, y, z)> degrees
|
||
style fill, wireframe, shading (Flat | VertexNormal | None), opacity
|
||
visibility visible, occlusion (Above | Below)
|
||
camera linked to the viewport camera, or an independent orbit
|
||
```
|
||
|
||
`ImportedGltf` stores an asset key and a hash, never the mesh. A missing asset degrades to
|
||
"asset missing" and the layer keeps its pose and transform. That is the same shape the source-
|
||
art binding already uses in `:reimport`: the mapping survives the pixels going away.
|
||
|
||
The built-in mannequin is procedural: a joint hierarchy plus primitive volumes. No bundled
|
||
asset, so no licensing question, and it is deterministic — the same proportions always produce
|
||
the same triangles, which a test can pin.
|
||
|
||
Joint set, minimum viable:
|
||
|
||
```
|
||
pelvis, spine, chest, neck, head
|
||
shoulder_l, elbow_l, wrist_l
|
||
hip_l, knee_l, ankle_l
|
||
shoulder_r, elbow_r, wrist_r
|
||
hip_r, knee_r, ankle_r
|
||
```
|
||
|
||
17 joints. Each has an anatomical clamp, so a pose cannot be built that a body cannot hold, and
|
||
the clamp is data the UI can show as a slider range rather than a silent rejection.
|
||
|
||
## Camera
|
||
|
||
`ReferenceCamera(target, yaw, pitch, zoom, projection)`, pure, returning a new instance,
|
||
touching no GL — the same discipline as `ViewportCamera`, so the maths unit-tests without a
|
||
context.
|
||
|
||
Orthographic by default. Perspective is an option with a focal length, because foreshortening —
|
||
the use that justifies the feature — is a perspective phenomenon. Orthographic is the default
|
||
because it agrees with the 2D plane exactly, and a reference that disagrees with the drawing's
|
||
projection is wrong in a way the user cannot see.
|
||
|
||
Two clamps, both found by the harness rather than assumed:
|
||
|
||
- `pitch` must be clamped to the open interval (−89.9°, +89.9°). At ±90° the camera looks along
|
||
the world up axis and the screen-right basis vector becomes a zero vector. The projection
|
||
still returns a number, so the failure is silent: the model collapses to a line and nothing
|
||
reports it. The clamp has to be in the camera, not in the UI drag handler.
|
||
- `zoom` reuses `ViewportCamera.MIN_ZOOM` / `MAX_ZOOM`.
|
||
|
||
Camera linking: when linked, the reference camera's target and zoom follow the viewport camera,
|
||
and only yaw and pitch are its own. When unlinked it is fully independent. Linked is the
|
||
default, because the useful case is "rotate the reference without losing the drawing".
|
||
|
||
## Draw path
|
||
|
||
The seam has no depth concept today. `RenderPipelineSpec` is `(purpose, blend, cullBackFaces)`;
|
||
`RenderTargetSpec` has no depth attachment; `PipelinePurpose` is closed at six values, none of
|
||
which is a 3D mesh. `MeshSpec.restPositions` is interleaved x,y.
|
||
|
||
Four additions, in `docs/seams.md`. They are not optional and they are not local: adding a
|
||
`PipelinePurpose` value forces every backend to answer for it, which is the stated intent of
|
||
that enum being closed. So this feature cannot land in `:render` alone.
|
||
|
||
Draw order:
|
||
|
||
- `Above` — draw after the puppet, no depth test. Always correct, no depth buffer needed.
|
||
Default.
|
||
- `Below` — draw before the puppet, no depth test.
|
||
- `Depth-sorted` — the mannequin occludes the puppet properly. Deferred: it needs the puppet to
|
||
write depth from drawable Z, and the puppet path has no depth today.
|
||
|
||
The reference layer must stay outside the layer-composite path. The composite samples
|
||
screen-space textures and blends in-shader with blending disabled; it has no depth input, so a
|
||
depth-tested draw composited through it would blend against pixels the depth test should have
|
||
rejected. Keep it a direct draw.
|
||
|
||
## Pose
|
||
|
||
Pose is a map of joint id to Euler triple, applied as `T(offset) · R(euler)` per joint, world =
|
||
parent world · local. Verified invariants in the harness:
|
||
|
||
- bone length is pose-invariant,
|
||
- a joint's rotation does not move its own origin,
|
||
- the rotation block stays orthonormal at scale 1, which is what skinning depends on.
|
||
|
||
Pose presets reuse the storage shape already planned for `TODO.md` § Pose Palette/Library —
|
||
Cubism 5.4's "Model state set" is a pose library, and the same container holds a mannequin pose
|
||
with no format work.
|
||
|
||
Skinning for imported glTF is CPU-side: joint matrices times weights, summed. Verified: a
|
||
70/30 blend lands between the two poses. CPU first matches the codebase's existing shape
|
||
(CPU deform eval as the oracle, GPU as the fast path), and it gives a testable oracle before
|
||
any GPU work.
|
||
|
||
## Persistence
|
||
|
||
A new UMA entry, `reference`. It stores the source, transform, pose, style, visibility, and
|
||
camera link. It does not store geometry — geometry comes from the asset or from the procedural
|
||
generator, both reproducible.
|
||
|
||
The entry must be additive and ignorable: a reader that does not know `reference` skips it and
|
||
the document still opens. Mirror `UmaEntrySchemaTest.kt` for the gate.
|
||
|
||
## UI
|
||
|
||
- Viewport header → Overlays toggle, which is already planned (`TODO.md` § Overlays Toggle) and
|
||
currently lists General Information, Wireframe, Grid, 2D Cursor. Add Reference Layer there.
|
||
- Properties panel → a Reference section.
|
||
- Commands through `CommandRegistry`, not hardcoded in a widget: `reference.import`,
|
||
`reference.reset-pose`, `reference.fit`, `reference.toggle-visibility`,
|
||
`reference.link-camera`, `reference.clear`.
|
||
- Settings keys, dotted namespace: `view.referenceLayer.style`, `.opacity`, `.wireframe`,
|
||
`.defaultHeightCm`, `.maxTriangles`, `.linkCamera`.
|
||
|
||
## Tests
|
||
|
||
Pure, `commonTest`:
|
||
|
||
- GLB container rules: header, chunk types, padding, truncation, accessor alignment.
|
||
- FK invariants above.
|
||
- Camera basis orthonormality, agreement with `ViewportCamera` at yaw 0 pitch 0, pitch clamp.
|
||
|
||
GL-gated, `jvmTest`, shaped like `WorldAxisLinesTest.kt`: a flat-shaded triangle at a known
|
||
camera must rasterize the expected pixels, and the depth test must reject a triangle behind it.
|
||
Skips in a display-less environment via a JUnit assumption, so the run reports SKIPPED rather
|
||
than a green pass that asserted nothing.
|
||
|
||
Tolerance, not bit-exactness, for anything with trig. The codebase's determinism rule is real,
|
||
but FK and the camera cannot be bit-identical across targets; the tests must say so explicitly
|
||
rather than assert exact floats and fail on a different machine.
|
||
|
||
## Risks
|
||
|
||
1. **Export contamination.** The reference layer must never reach the CMO3 writer. The
|
||
byte-identical round-trip gate is the thing that catches this, and it only catches it if the
|
||
gate is run with a reference layer present. Add that case.
|
||
2. **Closed enum cost.** `PipelinePurpose` is deliberately closed so a new pipeline forces every
|
||
backend to answer. The GLES and Metal devices are stubs, so this feature is desktop-only
|
||
until the GLES device exists (`TODO.md` § Deferred). That is fine, but the stubs must answer
|
||
explicitly rather than throw.
|
||
3. **Silent degeneracy.** The pitch collapse is the clearest example: a projection that returns
|
||
a number instead of failing. Any clamp added here belongs in the pure model, not in the
|
||
input handler.
|
||
4. **Memory.** An imported glTF can be 100k triangles. On a tablet that is not a reference
|
||
layer, it is a stall. Cap at import, report the cap as a notice, do not silently drop
|
||
primitives.
|
||
5. **Scope creep.** The feature's natural next step is a real 3D viewport, which is a different
|
||
project. The non-goals section exists to keep this one from becoming its first installment.
|