Camera and reader rules corrected against a real file

The depth row maps the slab to the full clip range, not half of it, so the CPU sort and the GPU
depth test agree on which fragment is nearer. The screen-right axis is the unit horizontal: built
as forward x worldUp it carries a factor of cos(pitch) and compresses screen X as the camera tilts,
so the pitch clamp is the edge-on guard rather than a fix for a zero vector.

MAT4 is an accessor type and count is the number of elements, not floats, which is the shape a
skin's inverse bind matrices actually have. node.matrix is column-major while Mat4 is row-major.
A skin's joints must be resolved by node index, because CesiumMan.glb keeps its skeleton outside
the scene root.
This commit is contained in:
litruv
2026-11-03 21:04:53 +11:00
parent 4eb583b5f3
commit 0894e762f9
2 changed files with 95 additions and 18 deletions
+63 -5
View File
@@ -69,6 +69,17 @@ Two rules that have to be stated, because they are not obvious from the API:
must not be composited through a blend-mode layer; the blend would run against pixels the must not be composited through a blend-mode layer; the blend would run against pixels the
depth test should have rejected. The reference layer draws directly. depth test should have rejected. The reference layer draws directly.
Three more rules the implementation settled:
- The depth clear follows the pass's `loadAction` and only runs when the target has a depth
attachment. Depth state is bound with the program, like blend and cull, and the write mask stays at
its default.
- `drawReferenceMesh` and `drawReferenceWireframe` check the pipeline's topology against the draw, so
a spec and a draw call cannot disagree.
- `DepthFunc.Always` is what the wireframe uses, so edges show over the mesh. A coplanar wireframe
loses to strict `Less` and proves nothing, which is why the depth proof in the GL test uses two
screen-overlapping squares at different depths instead of a wireframe-versus-mesh comparison.
## 3. `PipelinePurpose` additions ## 3. `PipelinePurpose` additions
```kotlin ```kotlin
@@ -94,18 +105,36 @@ explicitly, not throw.
```kotlin ```kotlin
public data class WorldToClip(val matrix: FloatArray) // 16 floats, row-major public data class WorldToClip(val matrix: FloatArray) // 16 floats, row-major
public data class DepthRange(val near: Float, val far: Float)
``` ```
```kotlin ```kotlin
fun setClipCamera(worldToClip: WorldToClip, depthRange: DepthRange, screenTexWidth: Int, screenTexHeight: Int) fun setClipCamera(worldToClip: WorldToClip)
``` ```
`DepthRange` was in this proposal and is gone: the near/far mapping is baked into the matrix, so a
separate depth uniform would be dead state sent to a program that does not declare it. The screen
size is gone too — the orthographic scales are already in the matrix.
The depth row maps the slab to the full clip range `[-1, 1]`, not to `[0, 1]`. Mapping to `[0, 1]`
throws away half the depth precision for nothing, and it makes the CPU sort and the GPU test
disagree about which fragment is nearer. `ReferenceCamera.project` reports window depth
`(clipZ + 1) / 2` in `[0, 1]`, so a smaller value is nearer in both paths. Geometry outside the
slab is clipped, so `near` has to cover the model's extent along the view direction.
`setCamera(worldToNdc, ...)` stays. `WorldToNdc` is its Z = 0 slice, and the two must agree `setCamera(worldToNdc, ...)` stays. `WorldToNdc` is its Z = 0 slice, and the two must agree
exactly at yaw = pitch = 0 — that agreement is verified in `verify/verify.py` and is what keeps exactly at yaw = pitch = 0 — that agreement is verified in `verify/verify.py` and is what keeps
the reference layer aligned with the drawing plane when nothing is rotated. the reference layer aligned with the drawing plane when nothing is rotated.
Two traps the implementation hit:
- `Mat4` is row-major because commonMain supplies values and the layout is the backend's, so the GL
marshaller transposes on upload. Applying the columns instead of the rows is the transpose, and it
is invisible at zero angles where the matrix is diagonal.
- The screen-right axis must be the unit horizontal, not `forward x worldUp`. That cross product
carries a factor of `cos(pitch)` and compresses screen X as the camera tilts, reaching exactly zero
at pitch 90. The pitch clamp is a usability guard — at pitch 90 the drawing plane is edge-on — not
a fix for a zero vector.
## 5. A 3D mesh upload ## 5. A 3D mesh upload
`MeshSpec.restPositions` is interleaved x,y. A reference mesh needs three components and has no `MeshSpec.restPositions` is interleaved x,y. A reference mesh needs three components and has no
@@ -115,14 +144,23 @@ UVs, no glue attributes, and no deform inputs.
public class ReferenceMeshSpec( public class ReferenceMeshSpec(
val positions: FloatArray, // interleaved x, y, z val positions: FloatArray, // interleaved x, y, z
val indices: IntArray, val indices: IntArray,
val normals: FloatArray?, // interleaved x, y, z, or null for flat shading val wireframeIndices: IntArray, // triangle edges as line pairs, built at upload
val wireframeIndices: IntArray?, // triangle edges as line pairs, built at upload
) )
``` ```
Building `wireframeIndices` at upload rather than per frame is the point: the edge set is Building `wireframeIndices` at upload rather than per frame is the point: the edge set is
static, and a per-frame rebuild would be the most expensive thing in the frame. static, and a per-frame rebuild would be the most expensive thing in the frame.
As implemented, `normals` is gone and `wireframeIndices` is non-null. The flat program reads no
attribute 1, so uploading normals would allocate a buffer nothing reads; and a mesh drawn as a
wireframe has to have edges. Shading is a later PR.
A VAO owns its element-array binding, so the triangle index buffer and the edge index buffer cannot
share one VAO. The reference mesh gets a second VAO sharing the position VBO.
Per-pose position upload reuses `updateMeshPositions` — same buffer kind, only the component count
differs.
## 6. Draw calls ## 6. Draw calls
```kotlin ```kotlin
@@ -135,6 +173,26 @@ fun drawReferenceWireframe(mesh: GpuMesh, fragment: FragmentUniforms)
reference layer needs. It does not need `useTexture`, `useMask`, or the multiply/screen colors. reference layer needs. It does not need `useTexture`, `useMask`, or the multiply/screen colors.
Reusing the struct is fine; the marshaller reads what it reads. Reusing the struct is fine; the marshaller reads what it reads.
## 7. What a real file taught the reader
The fixtures stayed green while `CesiumMan.glb` failed. Each of these is a rule the fixtures asserted
in the wrong direction.
- `MAT4` is an accessor type. `count` is the number of elements, not the number of floats, so a skin's
inverse bind matrices are one `MAT4` Float32 accessor whose count is the number of joints, and the
format allows the count to exceed the joints. The reader required `SCALAR` and rejected the file.
- `node.matrix` is column-major. `Mat4` is row-major. Applying the sixteen values as stored applies the
transpose, and a transpose is invisible whenever the matrix is diagonal.
- glTF is Y-up, Umamo is Z-up. `CesiumMan.glb`'s root node is named `Z_UP` and carries the conversion
matrix. The reader does not convert axes; the caller decides which axes an imported model arrives in.
- Joints have to be resolved by node index. In that file the scene root holds only the mesh node, and the
nineteen joints hang off a skeleton node the scene does not reach. A reader that walked the scene graph
would find no joints and draw the mesh in its bind pose with nothing saying so.
- A primitive's mode can be a line mode — that file declares `LINE_LOOP`. The reader reports the mode; the
render side has to say what a non-triangle primitive means.
- `JOINTS_0` is `VEC4` `UNSIGNED_SHORT`, so skinning reads components, not floats. `readFloats` refuses it,
which is why `readComponents` is public.
## What does not change ## What does not change
- The deform eval, pose resolution, glue, and render order are untouched. - The deform eval, pose resolution, glue, and render order are untouched.
+32 -13
View File
@@ -218,7 +218,10 @@ check('70/30 blend lands between the two poses', [round(v, 6) for v in skinned],
def basis(yaw_deg, pitch_deg): def basis(yaw_deg, pitch_deg):
y, p = math.radians(yaw_deg), math.radians(pitch_deg) y, p = math.radians(yaw_deg), math.radians(pitch_deg)
forward = (-math.cos(p) * math.sin(y), math.cos(p) * math.cos(y), -math.sin(p)) forward = (-math.cos(p) * math.sin(y), math.cos(p) * math.cos(y), -math.sin(p))
right = (math.cos(p) * math.cos(y), math.cos(p) * math.sin(y), 0.0) # The unit horizontal, not forward x worldUp. That cross product carries a factor of cos(pitch)
# and shrinks to a zero vector at pitch 90, which would scale screen X by cos(pitch) as the camera
# tilts - a silent compression, not an error.
right = (math.cos(y), math.sin(y), 0.0)
up = (-math.sin(p) * math.sin(y), math.sin(p) * math.cos(y), math.cos(p)) up = (-math.sin(p) * math.sin(y), math.sin(p) * math.cos(y), math.cos(p))
return forward, right, up return forward, right, up
@@ -261,14 +264,15 @@ approx('basis stays orthonormal at yaw 37 pitch 21',
round(f[0] * u[0] + f[1] * u[1] + f[2] * u[2], 9), round(f[0] * u[0] + f[1] * u[1] + f[2] * u[2], 9),
round(r[0] * u[0] + r[1] * u[1] + r[2] * u[2], 9)], [0.0, 0.0, 0.0]) round(r[0] * u[0] + r[1] * u[1] + r[2] * u[2], 9)], [0.0, 0.0, 0.0])
# Real degeneracy: at pitch == +/-90 the camera looks along the world up axis, so the # At pitch == +/-90 the camera looks along the world up axis, which is the drawing plane's normal,
# screen-right axis is undefined. The camera must clamp pitch to the open interval and # so the puppet is seen edge-on and the reference layer collapses to a line. Nothing reports it -
# the UI must not allow a drag that reaches it. # the projection still returns a number - so the clamp belongs in the model, not in the drag handler.
# The right axis is the unit horizontal, so it does not collapse here; building it as
# forward x worldUp would have made it cos(pitch), exactly zero at 90.
f, r, u = basis(0, 90) f, r, u = basis(0, 90)
check('pitch 90 collapses the right axis (must be clamped)', round(math.sqrt(sum(v * v for v in r)), 9), 0.0) check('right stays unit length at pitch 90', round(math.sqrt(sum(v * v for v in r)), 9), 1.0)
check('pitch 90 puts the view along the world up axis', [round(v, 9) for v in f], [0.0, 0.0, -1.0])
f, r, u = basis(0, 89.9) check('and the drawing plane is edge-on: up projects to nothing', round(u[2], 9), 0.0)
approx('pitch 89.9 still gives a usable right axis', [round(v, 6) for v in r], (0.001745, 0.0, 0.0))
print() print()
if FAILURES: if FAILURES:
@@ -283,7 +287,10 @@ print('all checks pass')
def ortho_matrix(target, yaw, pitch, zoom, w, h, near, far): def ortho_matrix(target, yaw, pitch, zoom, w, h, near, far):
forward, right, up = basis(yaw, pitch) forward, right, up = basis(yaw, pitch)
sx, sy = 2 * zoom / w, 2 * zoom / h sx, sy = 2 * zoom / w, 2 * zoom / h
dz = 1.0 / (far - near) # near and far are distances from the target, so the slab runs from -near (toward the viewer) to
# +far. The depth row maps that slab to the full clip range [-1, 1]; mapping it to [0, 1] would
# throw away half the precision for nothing.
dz = 2.0 / (near + far)
def dot(a, b): def dot(a, b):
return a[0] * b[0] + a[1] * b[1] + a[2] * b[2] return a[0] * b[0] + a[1] * b[1] + a[2] * b[2]
m = [0.0] * 16 m = [0.0] * 16
@@ -291,8 +298,8 @@ def ortho_matrix(target, yaw, pitch, zoom, w, h, near, far):
for c in range(3): for c in range(3):
m[r * 4 + c] = axis[c] * scale m[r * 4 + c] = axis[c] * scale
m[r * 4 + 3] = -dot(axis, target) * scale m[r * 4 + 3] = -dot(axis, target) * scale
m[2 * 4 + 3] += near * dz - 1.0
m[15] = 1.0 m[15] = 1.0
m[2 * 4 + 3] -= near * dz
return m return m
@@ -300,12 +307,24 @@ def apply_matrix(m, p):
return tuple(m[r * 4 + 0] * p[0] + m[r * 4 + 1] * p[1] + m[r * 4 + 2] * p[2] + m[r * 4 + 3] for r in range(3)) return tuple(m[r * 4 + 0] * p[0] + m[r * 4 + 1] * p[1] + m[r * 4 + 2] * p[2] + m[r * 4 + 3] for r in range(3))
def window_depth(clip_z):
return (clip_z + 1.0) / 2.0
NEAR, FAR = 10.0, 30.0
for point, yaw, pitch in (((1, 0, 0), 0, 0), ((0, 0, 1), 0, 0), ((0, 1, 0), 90, 0), ((0.3, 0.4, 0.5), 37, 21)): for point, yaw, pitch in (((1, 0, 0), 0, 0), ((0, 0, 1), 0, 0), ((0, 1, 0), 90, 0), ((0.3, 0.4, 0.5), 37, 21)):
direct, _ = project(point, (0, 0, 0), yaw, pitch, 1, 400, 400) direct, _ = project(point, (0, 0, 0), yaw, pitch, 1, 400, 400)
m = ortho_matrix((0, 0, 0), yaw, pitch, 1, 400, 400, -10.0, 10.0) m = ortho_matrix((0, 0, 0), yaw, pitch, 1, 400, 400, NEAR, FAR)
via_matrix = apply_matrix(m, point) via_matrix = apply_matrix(m, point)
depth_ok = abs(via_matrix[2] - (direct[2] + 10) / 20) < 1e-9 depth_ok = abs(window_depth(via_matrix[2]) - (direct[2] + NEAR) / (NEAR + FAR)) < 1e-9
xy_ok = all(abs(a - b) < 1e-9 for a, b in zip(via_matrix[:2], direct[:2])) xy_ok = all(abs(a - b) < 1e-9 for a, b in zip(via_matrix[:2], direct[:2]))
check(f'matrix form agrees with the direct form at yaw {yaw} pitch {pitch}', (round(xy_ok), round(depth_ok)), (1, 1)) check(f'matrix form agrees with the direct form at yaw {yaw} pitch {pitch}', (round(xy_ok), round(depth_ok)), (1, 1))
check('depth maps near to 0 and far to 1', [round(v, 6) for v in apply_matrix(ortho_matrix((0, 0, 0), 0, 0, 1, 400, 400, -10.0, 10.0), (0, -10, 0))], [0.0, 0.0, 0.0]) check('the near end of the slab maps to window depth 0', round(window_depth(apply_matrix(ortho_matrix((0, 0, 0), 0, 0, 1, 400, 400, NEAR, FAR), (0, -NEAR, 0))[2]), 6), 0.0)
check('the far end of the slab maps to window depth 1', round(window_depth(apply_matrix(ortho_matrix((0, 0, 0), 0, 0, 1, 400, 400, NEAR, FAR), (0, FAR, 0))[2]), 6), 1.0)
# Applying the columns instead of the rows is the transpose. It is invisible at zero angles, where the
# matrix is diagonal, so the check has to run at a rotated camera.
m = ortho_matrix((0, 0, 0), 37, 21, 1, 400, 400, NEAR, FAR)
check('the matrix is applied by rows, not columns', round(m[1], 9) != round(m[4], 9), True)