Files
litruv 4eb583b5f3 Design doc and reference implementation sketch for the reference layer
Design for a poseable mannequin plus glTF/GLB import overlay in Umamo:
world axes, orbit camera, draw path, UMA persistence, UI, risks.

docs/seams.md lists the RenderDevice additions the feature needs:
PrimitiveTopology, DepthState, depthAttachment, WorldToClip,
ReferenceMeshSpec.

sketch/ mirrors the algorithms in Kotlin. verify/verify.py proves them:
30 checks, all passing. Two findings: accessor byteOffset must be a
multiple of the component size, and the screen-right basis collapses to a
zero vector at pitch +/-90 degrees, so pitch is clamped to an open
interval.
2026-11-03 18:14:07 +11:00

205 lines
10 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 3D model reference layer
## What it is for
A drawing aid that sits in the viewport over the puppet. Five uses, in the order people
actually reach for them:
1. **Proportion check.** Is this head too big for this body. Umamo has no body, so today the
answer comes from memory or a photo pinned next to the window.
2. **Anatomy reference.** Where the elbow actually is in a pose, where the shoulder joint sits,
how a ribcage turns.
3. **Foreshortening.** A hand coming toward the camera. A 2D reference cannot show this; a
3D one can, and this is the one use that justifies the feature existing at all.
4. **Scale reference.** "This puppet is 160 cm tall." Everything else follows from that.
5. **Line-art underlay.** Pose the mannequin, lower its opacity, draw over it.
It is not a 3D modeler, and the distinction matters for every decision below.
## Non-goals
- No 3D puppet. The puppet stays a 2D deforming mesh set.
- No glTF export. `TODO.md` already lists glTF export as a far-future item; it is a different
problem and shares nothing with this except the word glTF.
- No lighting, no material system, no texture sampling on the reference layer. Flat fill,
wireframe, or vertex-normal shading. Three options, no more.
- No animation playback on the reference layer. Pose is a static state.
- No physics, no cloth, no bendy bones. `TODO.md` § Future Feature Wishes already wants a real
skeleton for the puppet; that is a separate project and this one must not become its
accidental first installment.
- Nothing from the reference layer reaches CMO3 or MOC3 export. It is editor-only. See § Risks.
## World-space convention
Umamo's world space is X right, Y depth, Z up. `PuppetModel.worldOriginX` and `worldOriginZ`
already exist, and the viewport already draws the X and Z axis lines through the world origin
(`module/render/src/jvmTest/.../gl/WorldAxisLinesTest.kt`).
The reference layer uses that same frame. No new axis convention, no conversion. A mannequin
standing at the world origin is already aligned with the drawing, and at yaw = pitch = 0 the
reference camera must agree exactly with `ViewportCamera.worldToNdc`. That agreement is the
anchor of the whole design: the reference layer at rest sits exactly in the drawing plane, and
rotating it is the only thing that separates it from the puppet.
Verified in `verify/verify.py`: at yaw 0 pitch 0 the camera basis is forward +Y, right +X,
up +Z, and a point one world unit right projects to NDC x = 2·zoom/viewportWidth, which is
`ViewportCamera.worldToNdc`'s value.
Units are model pixels: one world unit is one model pixel. A mannequin therefore needs a
scale stated in human units, not pixels. `referenceHeightCm` (default 160) plus the document's
puppet height in world units gives the scale, and that is what makes the proportion check
possible at all — a head is about 1/7.5 of standing height, and the mannequin encodes that as
data rather than as a guess.
## Data model
```
ReferenceLayer
source BuiltInMannequin(proportions) | ImportedGltf(assetKey, sceneIndex)
transform position (x, y, z), scale, rotation (yaw, pitch, roll)
pose Map<JointId, Euler(x, y, z)> degrees
style fill, wireframe, shading (Flat | VertexNormal | None), opacity
visibility visible, occlusion (Above | Below)
camera linked to the viewport camera, or an independent orbit
```
`ImportedGltf` stores an asset key and a hash, never the mesh. A missing asset degrades to
"asset missing" and the layer keeps its pose and transform. That is the same shape the source-
art binding already uses in `:reimport`: the mapping survives the pixels going away.
The built-in mannequin is procedural: a joint hierarchy plus primitive volumes. No bundled
asset, so no licensing question, and it is deterministic — the same proportions always produce
the same triangles, which a test can pin.
Joint set, minimum viable:
```
pelvis, spine, chest, neck, head
shoulder_l, elbow_l, wrist_l
hip_l, knee_l, ankle_l
shoulder_r, elbow_r, wrist_r
hip_r, knee_r, ankle_r
```
17 joints. Each has an anatomical clamp, so a pose cannot be built that a body cannot hold, and
the clamp is data the UI can show as a slider range rather than a silent rejection.
## Camera
`ReferenceCamera(target, yaw, pitch, zoom, projection)`, pure, returning a new instance,
touching no GL — the same discipline as `ViewportCamera`, so the maths unit-tests without a
context.
Orthographic by default. Perspective is an option with a focal length, because foreshortening —
the use that justifies the feature — is a perspective phenomenon. Orthographic is the default
because it agrees with the 2D plane exactly, and a reference that disagrees with the drawing's
projection is wrong in a way the user cannot see.
Two clamps, both found by the harness rather than assumed:
- `pitch` must be clamped to the open interval (−89.9°, +89.9°). At ±90° the camera looks along
the world up axis and the screen-right basis vector becomes a zero vector. The projection
still returns a number, so the failure is silent: the model collapses to a line and nothing
reports it. The clamp has to be in the camera, not in the UI drag handler.
- `zoom` reuses `ViewportCamera.MIN_ZOOM` / `MAX_ZOOM`.
Camera linking: when linked, the reference camera's target and zoom follow the viewport camera,
and only yaw and pitch are its own. When unlinked it is fully independent. Linked is the
default, because the useful case is "rotate the reference without losing the drawing".
## Draw path
The seam has no depth concept today. `RenderPipelineSpec` is `(purpose, blend, cullBackFaces)`;
`RenderTargetSpec` has no depth attachment; `PipelinePurpose` is closed at six values, none of
which is a 3D mesh. `MeshSpec.restPositions` is interleaved x,y.
Four additions, in `docs/seams.md`. They are not optional and they are not local: adding a
`PipelinePurpose` value forces every backend to answer for it, which is the stated intent of
that enum being closed. So this feature cannot land in `:render` alone.
Draw order:
- `Above` — draw after the puppet, no depth test. Always correct, no depth buffer needed.
Default.
- `Below` — draw before the puppet, no depth test.
- `Depth-sorted` — the mannequin occludes the puppet properly. Deferred: it needs the puppet to
write depth from drawable Z, and the puppet path has no depth today.
The reference layer must stay outside the layer-composite path. The composite samples
screen-space textures and blends in-shader with blending disabled; it has no depth input, so a
depth-tested draw composited through it would blend against pixels the depth test should have
rejected. Keep it a direct draw.
## Pose
Pose is a map of joint id to Euler triple, applied as `T(offset) · R(euler)` per joint, world =
parent world · local. Verified invariants in the harness:
- bone length is pose-invariant,
- a joint's rotation does not move its own origin,
- the rotation block stays orthonormal at scale 1, which is what skinning depends on.
Pose presets reuse the storage shape already planned for `TODO.md` § Pose Palette/Library —
Cubism 5.4's "Model state set" is a pose library, and the same container holds a mannequin pose
with no format work.
Skinning for imported glTF is CPU-side: joint matrices times weights, summed. Verified: a
70/30 blend lands between the two poses. CPU first matches the codebase's existing shape
(CPU deform eval as the oracle, GPU as the fast path), and it gives a testable oracle before
any GPU work.
## Persistence
A new UMA entry, `reference`. It stores the source, transform, pose, style, visibility, and
camera link. It does not store geometry — geometry comes from the asset or from the procedural
generator, both reproducible.
The entry must be additive and ignorable: a reader that does not know `reference` skips it and
the document still opens. Mirror `UmaEntrySchemaTest.kt` for the gate.
## UI
- Viewport header → Overlays toggle, which is already planned (`TODO.md` § Overlays Toggle) and
currently lists General Information, Wireframe, Grid, 2D Cursor. Add Reference Layer there.
- Properties panel → a Reference section.
- Commands through `CommandRegistry`, not hardcoded in a widget: `reference.import`,
`reference.reset-pose`, `reference.fit`, `reference.toggle-visibility`,
`reference.link-camera`, `reference.clear`.
- Settings keys, dotted namespace: `view.referenceLayer.style`, `.opacity`, `.wireframe`,
`.defaultHeightCm`, `.maxTriangles`, `.linkCamera`.
## Tests
Pure, `commonTest`:
- GLB container rules: header, chunk types, padding, truncation, accessor alignment.
- FK invariants above.
- Camera basis orthonormality, agreement with `ViewportCamera` at yaw 0 pitch 0, pitch clamp.
GL-gated, `jvmTest`, shaped like `WorldAxisLinesTest.kt`: a flat-shaded triangle at a known
camera must rasterize the expected pixels, and the depth test must reject a triangle behind it.
Skips in a display-less environment via a JUnit assumption, so the run reports SKIPPED rather
than a green pass that asserted nothing.
Tolerance, not bit-exactness, for anything with trig. The codebase's determinism rule is real,
but FK and the camera cannot be bit-identical across targets; the tests must say so explicitly
rather than assert exact floats and fail on a different machine.
## Risks
1. **Export contamination.** The reference layer must never reach the CMO3 writer. The
byte-identical round-trip gate is the thing that catches this, and it only catches it if the
gate is run with a reference layer present. Add that case.
2. **Closed enum cost.** `PipelinePurpose` is deliberately closed so a new pipeline forces every
backend to answer. The GLES and Metal devices are stubs, so this feature is desktop-only
until the GLES device exists (`TODO.md` § Deferred). That is fine, but the stubs must answer
explicitly rather than throw.
3. **Silent degeneracy.** The pitch collapse is the clearest example: a projection that returns
a number instead of failing. Any clamp added here belongs in the pure model, not in the
input handler.
4. **Memory.** An imported glTF can be 100k triangles. On a tablet that is not a reference
layer, it is a stall. Cap at import, report the cap as a notice, do not silently drop
primitives.
5. **Scope creep.** The feature's natural next step is a real 3D viewport, which is a different
project. The non-goals section exists to keep this one from becoming its first installment.