Coordinate Systems and Poses

• Getting Started
• JavaScript
• Universal AR

The Zappar library models the 3D space of an AR experience using three transformations:

To render content relative to an anchor, the following transformation (the ModelViewProjection matrix) is computed, typically in the vertex shader:

mat4 modelViewProjection = projectionMatrix * inverseCameraPoseMatrix * anchorPoseMatrix;
gl_Position = modelViewProjection * vertexPosition;

The following sections show how to compute these various constituent transformation matrices.

Projection Matrix

When rendering an AR experience, it's important that the projection matrix used to render the virtual 3D content matches the parameters of the physical camera (e.g. focal length) being processed and displayed. The Zappar library provides a function to get these parameters for the current frame (herein referred to as the camera model) and a function to convert them into a projection matrix you can use during rendering:

let model = pipeline.cameraModel();
let projectionMatrix = Zappar.projectionMatrixFromCameraModel(model, renderWidth, renderHeight);

Pass the dimensions of your canvas for renderWidth, renderHeight and, if needed , the zNearand zFar clipping plane parameters. The resulting projectionMatrix is a Float32Array containing a 4x4 column-wise matrix that you can use directly as a uniform in your vertex shader.

You should call these functions every frame after your pipeline.frameUpdate() call, since the Zappar library may change the camera model over time as it learns more about the physical camera.

Camera Pose

The Zappar library provides multiple functions for obtaining a camera pose. Each function defines a different world space and thus differing behavior for the camera as the user moves their device in space and around any anchors that are being tracked. Each of the functions return a Float32Array containing a 4x4 column-major matrix.

The correct choice of camera pose will depend on your given use case and content. Here are some examples you might like to consider when choosing which is best for you:

• To have a light that always shines down from above the user, regardless of the angle of the device or anchors, use cameraPoseWithAttitude and simulate a light shining down the negative Y axis is world space.
• In an application with a physics simulation of stacked blocks, and with gravity pointing down the negative Y axis of world space, using cameraPoseWithOrigin would allow the blocks to rest on a tracked image regardless of how the image is held by the user, while using cameraPoseWithAttitude would allow the user to tip the blocks off the image by tilting it.

The matrices returned by these functions represent the transformation of the camera relative to a world space, but if you're forming a full ModelViewProjection matrix in order to render content, then you need to use the inverse of the camera transformation. For this purpose, the Zappar library provides a convenience function to compute the inverse:

let inverseCameraPoseMatrix = Zappar.invert(cameraPose);

Anchor Pose

Each of the tracking types provided by the Zappar library expose anchors with a function to obtain an anchor pose for a given camera pose, e.g.:

let cameraPoseMatrix = pipeline.cameraPoseDefault();
let anchorPoseMatrix = myFaceAnchor.pose(cameraPose);

It's best to use this structure, even if you're using cameraPoseWithOrigin and the anchor is forming the origin of your world space, like this:

let cameraPoseMatrix = pipeline.cameraPoseWithOrigin(myFaceAnchor.poseCameraRelative());
let anchorPoseMatrix = myFaceAnchor.pose(cameraPose);

This pose matrix forms the final transformation for a complete ModelViewProjection matrix for rendering:

mat4 modelViewProjection = projectionMatrix * inverseCameraPoseMatrix * anchorPoseMatrix;
gl_Position = modelViewProjection * vertexPosition;

The following sections gives more details about the various tracking types and their associated anchors:

• Image Tracking
• Instant World Tracking
• Face Tracking