Files
umamo-reference-layer/docs/spec.md
T
litruv 4eb583b5f3 Design doc and reference implementation sketch for the reference layer
Design for a poseable mannequin plus glTF/GLB import overlay in Umamo:
world axes, orbit camera, draw path, UMA persistence, UI, risks.

docs/seams.md lists the RenderDevice additions the feature needs:
PrimitiveTopology, DepthState, depthAttachment, WorldToClip,
ReferenceMeshSpec.

sketch/ mirrors the algorithms in Kotlin. verify/verify.py proves them:
30 checks, all passing. Two findings: accessor byteOffset must be a
multiple of the component size, and the screen-right basis collapses to a
zero vector at pitch +/-90 degrees, so pitch is clamped to an open
interval.
2026-11-03 18:14:07 +11:00

10 KiB
Raw Blame History

3D model reference layer

What it is for

A drawing aid that sits in the viewport over the puppet. Five uses, in the order people actually reach for them:

  1. Proportion check. Is this head too big for this body. Umamo has no body, so today the answer comes from memory or a photo pinned next to the window.
  2. Anatomy reference. Where the elbow actually is in a pose, where the shoulder joint sits, how a ribcage turns.
  3. Foreshortening. A hand coming toward the camera. A 2D reference cannot show this; a 3D one can, and this is the one use that justifies the feature existing at all.
  4. Scale reference. "This puppet is 160 cm tall." Everything else follows from that.
  5. Line-art underlay. Pose the mannequin, lower its opacity, draw over it.

It is not a 3D modeler, and the distinction matters for every decision below.

Non-goals

  • No 3D puppet. The puppet stays a 2D deforming mesh set.
  • No glTF export. TODO.md already lists glTF export as a far-future item; it is a different problem and shares nothing with this except the word glTF.
  • No lighting, no material system, no texture sampling on the reference layer. Flat fill, wireframe, or vertex-normal shading. Three options, no more.
  • No animation playback on the reference layer. Pose is a static state.
  • No physics, no cloth, no bendy bones. TODO.md § Future Feature Wishes already wants a real skeleton for the puppet; that is a separate project and this one must not become its accidental first installment.
  • Nothing from the reference layer reaches CMO3 or MOC3 export. It is editor-only. See § Risks.

World-space convention

Umamo's world space is X right, Y depth, Z up. PuppetModel.worldOriginX and worldOriginZ already exist, and the viewport already draws the X and Z axis lines through the world origin (module/render/src/jvmTest/.../gl/WorldAxisLinesTest.kt).

The reference layer uses that same frame. No new axis convention, no conversion. A mannequin standing at the world origin is already aligned with the drawing, and at yaw = pitch = 0 the reference camera must agree exactly with ViewportCamera.worldToNdc. That agreement is the anchor of the whole design: the reference layer at rest sits exactly in the drawing plane, and rotating it is the only thing that separates it from the puppet.

Verified in verify/verify.py: at yaw 0 pitch 0 the camera basis is forward +Y, right +X, up +Z, and a point one world unit right projects to NDC x = 2·zoom/viewportWidth, which is ViewportCamera.worldToNdc's value.

Units are model pixels: one world unit is one model pixel. A mannequin therefore needs a scale stated in human units, not pixels. referenceHeightCm (default 160) plus the document's puppet height in world units gives the scale, and that is what makes the proportion check possible at all — a head is about 1/7.5 of standing height, and the mannequin encodes that as data rather than as a guess.

Data model

ReferenceLayer
  source        BuiltInMannequin(proportions) | ImportedGltf(assetKey, sceneIndex)
  transform     position (x, y, z), scale, rotation (yaw, pitch, roll)
  pose          Map<JointId, Euler(x, y, z)>          degrees
  style         fill, wireframe, shading (Flat | VertexNormal | None), opacity
  visibility    visible, occlusion (Above | Below)
  camera        linked to the viewport camera, or an independent orbit

ImportedGltf stores an asset key and a hash, never the mesh. A missing asset degrades to "asset missing" and the layer keeps its pose and transform. That is the same shape the source- art binding already uses in :reimport: the mapping survives the pixels going away.

The built-in mannequin is procedural: a joint hierarchy plus primitive volumes. No bundled asset, so no licensing question, and it is deterministic — the same proportions always produce the same triangles, which a test can pin.

Joint set, minimum viable:

pelvis, spine, chest, neck, head
shoulder_l, elbow_l, wrist_l
hip_l, knee_l, ankle_l
shoulder_r, elbow_r, wrist_r
hip_r, knee_r, ankle_r

17 joints. Each has an anatomical clamp, so a pose cannot be built that a body cannot hold, and the clamp is data the UI can show as a slider range rather than a silent rejection.

Camera

ReferenceCamera(target, yaw, pitch, zoom, projection), pure, returning a new instance, touching no GL — the same discipline as ViewportCamera, so the maths unit-tests without a context.

Orthographic by default. Perspective is an option with a focal length, because foreshortening — the use that justifies the feature — is a perspective phenomenon. Orthographic is the default because it agrees with the 2D plane exactly, and a reference that disagrees with the drawing's projection is wrong in a way the user cannot see.

Two clamps, both found by the harness rather than assumed:

  • pitch must be clamped to the open interval (−89.9°, +89.9°). At ±90° the camera looks along the world up axis and the screen-right basis vector becomes a zero vector. The projection still returns a number, so the failure is silent: the model collapses to a line and nothing reports it. The clamp has to be in the camera, not in the UI drag handler.
  • zoom reuses ViewportCamera.MIN_ZOOM / MAX_ZOOM.

Camera linking: when linked, the reference camera's target and zoom follow the viewport camera, and only yaw and pitch are its own. When unlinked it is fully independent. Linked is the default, because the useful case is "rotate the reference without losing the drawing".

Draw path

The seam has no depth concept today. RenderPipelineSpec is (purpose, blend, cullBackFaces); RenderTargetSpec has no depth attachment; PipelinePurpose is closed at six values, none of which is a 3D mesh. MeshSpec.restPositions is interleaved x,y.

Four additions, in docs/seams.md. They are not optional and they are not local: adding a PipelinePurpose value forces every backend to answer for it, which is the stated intent of that enum being closed. So this feature cannot land in :render alone.

Draw order:

  • Above — draw after the puppet, no depth test. Always correct, no depth buffer needed. Default.
  • Below — draw before the puppet, no depth test.
  • Depth-sorted — the mannequin occludes the puppet properly. Deferred: it needs the puppet to write depth from drawable Z, and the puppet path has no depth today.

The reference layer must stay outside the layer-composite path. The composite samples screen-space textures and blends in-shader with blending disabled; it has no depth input, so a depth-tested draw composited through it would blend against pixels the depth test should have rejected. Keep it a direct draw.

Pose

Pose is a map of joint id to Euler triple, applied as T(offset) · R(euler) per joint, world = parent world · local. Verified invariants in the harness:

  • bone length is pose-invariant,
  • a joint's rotation does not move its own origin,
  • the rotation block stays orthonormal at scale 1, which is what skinning depends on.

Pose presets reuse the storage shape already planned for TODO.md § Pose Palette/Library — Cubism 5.4's "Model state set" is a pose library, and the same container holds a mannequin pose with no format work.

Skinning for imported glTF is CPU-side: joint matrices times weights, summed. Verified: a 70/30 blend lands between the two poses. CPU first matches the codebase's existing shape (CPU deform eval as the oracle, GPU as the fast path), and it gives a testable oracle before any GPU work.

Persistence

A new UMA entry, reference. It stores the source, transform, pose, style, visibility, and camera link. It does not store geometry — geometry comes from the asset or from the procedural generator, both reproducible.

The entry must be additive and ignorable: a reader that does not know reference skips it and the document still opens. Mirror UmaEntrySchemaTest.kt for the gate.

UI

  • Viewport header → Overlays toggle, which is already planned (TODO.md § Overlays Toggle) and currently lists General Information, Wireframe, Grid, 2D Cursor. Add Reference Layer there.
  • Properties panel → a Reference section.
  • Commands through CommandRegistry, not hardcoded in a widget: reference.import, reference.reset-pose, reference.fit, reference.toggle-visibility, reference.link-camera, reference.clear.
  • Settings keys, dotted namespace: view.referenceLayer.style, .opacity, .wireframe, .defaultHeightCm, .maxTriangles, .linkCamera.

Tests

Pure, commonTest:

  • GLB container rules: header, chunk types, padding, truncation, accessor alignment.
  • FK invariants above.
  • Camera basis orthonormality, agreement with ViewportCamera at yaw 0 pitch 0, pitch clamp.

GL-gated, jvmTest, shaped like WorldAxisLinesTest.kt: a flat-shaded triangle at a known camera must rasterize the expected pixels, and the depth test must reject a triangle behind it. Skips in a display-less environment via a JUnit assumption, so the run reports SKIPPED rather than a green pass that asserted nothing.

Tolerance, not bit-exactness, for anything with trig. The codebase's determinism rule is real, but FK and the camera cannot be bit-identical across targets; the tests must say so explicitly rather than assert exact floats and fail on a different machine.

Risks

  1. Export contamination. The reference layer must never reach the CMO3 writer. The byte-identical round-trip gate is the thing that catches this, and it only catches it if the gate is run with a reference layer present. Add that case.
  2. Closed enum cost. PipelinePurpose is deliberately closed so a new pipeline forces every backend to answer. The GLES and Metal devices are stubs, so this feature is desktop-only until the GLES device exists (TODO.md § Deferred). That is fine, but the stubs must answer explicitly rather than throw.
  3. Silent degeneracy. The pitch collapse is the clearest example: a projection that returns a number instead of failing. Any clamp added here belongs in the pure model, not in the input handler.
  4. Memory. An imported glTF can be 100k triangles. On a tablet that is not a reference layer, it is a stall. Cap at import, report the cap as a notice, do not silently drop primitives.
  5. Scope creep. The feature's natural next step is a real 3D viewport, which is a different project. The non-goals section exists to keep this one from becoming its first installment.