Camera Setup

The ZapparThree library needs to use your WebGL context in order to process camera frames. You can set it when your page loads using the following function:

ZapparThree.glContextSet(renderer.getContext());

Constructing the Camera

The library provides a camera object that you can use instead of a three.js PerspectiveCamera. It's constructed like this:

let camera = new ZapparThree.Camera();

The camera object provides a THREE.Texture object containing the camera feed. You can use this texture however you wish but the easiest way to show the camera feed behind your content is to set it as your scene's background:

scene.background = camera.backgroundTexture;

Options

Options may be passed into the ZapparThree.Camera constructor.

Custom Video Devices

let camera = new ZapparThree.Camera(
    {
        rearCameraSource:'csO9c0YpAf274OuCPUA53CNE0YHlIr2yXCi+SqfBZZ8=',
        userCameraSource: 'RKxXByjnabbADGQNNZqLVLdmXlS0YkETYCIbg+XxnvM='
    }
);

Clipping Planes

let camera = new ZapparThree.Camera(
    { // These values are defaults.
        zNear: 0.1,
        zFar: 100
    }
);

Permissions

The library needs to ask the user for permission to access the camera and motion sensors on the device.

To do this, you can use the following function to show a built-in UI informing the user of the need and providing a button to trigger the browser's permission prompts. The function returns a promise that lets you know if the user granted the permissions or not.

// Show Zappar's built-in UI to request camera permissions
ZapparThree.permissionRequestUI().then(granted => {
    if (granted) {
        // User granted the permissions so start the camera
        camera.start();
    } else {
        // User denied the permissions so show Zappar's built-in 'permission denied' UI
        ZapparThree.permissionDeniedUI();
    }
});

If you'd rather show your own permissions UI, you can use the following function to trigger the browser's permission prompts directly. The function returns a promise that resolves to true if the user granted all the necessary permissions, otherwise false. Please note - due to browser restrictions, this function must be called from within a user event, e.g. in the event handler of a button click.

ZapparThree.permissionRequest().then(granted => {
    if (granted) {
        // User granted the permissions so start the camera
    } else {
        // User denied the permissions
        // You can show your own 'permission denied' UI here or use Zappar's built-in one
        ZapparThree.permissionDeniedUI();
    }
});

Starting the Camera

Once the user has granted the necessary permissions, you can start the rear-facing camera on the device with the following function on your camera object:

camera.start();

If you'd like to start the user-facing 'selfie' camera, pass true:

camera.start(true);

To switch between the front- and rear-facing cameras during your experience, just call camera.start(...) again passing either true or false as appropriate.

User-facing cameras are normally shown mirrored to users and so the camera object provides a number of modes to support this:

  • ZapparThree.CameraMirrorMode.Poses: this mode mirrors the background camera texture and ensures content still appears correctly tracked. In this mode your content itself isn't flipped, so any text in your tracked content doesn't appear mirrored. This is the default mode for the user-facing camera.
  • ZapparThree.CameraMirrorMode.CSS: in this mode, the Zappar camera applies a scaleX(-1) CSS transform to your whole canvas. This way both the camera and your content appear mirrored.
  • ZapparThree.CameraMirrorMode.None: no mirroring is performed. This is the default mode for the rear-facing camera.

The userCameraMirrorMode and rearCameraMirrorMode parameters set which mode the camera object will use for each camera:

camera.userCameraMirrorMode = ZapparThree.CameraMirrorMode.CSS;

Processing Camera Frames

Call the following function once an animation frame (e.g. during your requestAnimationFrame function) in order to process incoming camera frames:

camera.updateFrame(renderer);

Realtime Camera-based Reflections

The SDK provides an automatically generated environment map that's useful if you're using materials that support reflections (e.g. MeshStandardMaterial, MeshPhysicalMaterial). The map uses the camera feed to create an approximate environment that can add some realism to your scene.

To use the map, first construct an instance:

const envMap = new ZapparThree.CameraEnvironmentMap();

Attach the map to your scene to affect all relevant materials:

scene.environment = envMap.environmentMap;

Or attach it to specific materials, if you prefer:

material.envMap = envMap.environmentMap;

Finally, call update(...) on the map each frame, between updating the camera and rendering the scene:

// Update your Zappar camera
camera.updateFrame(renderer);

// Update your Zappar environment map
envMap.update(renderer, camera);

// Update your scene
renderer.render(scene, camera);

Camera Pose

The Zappar library provides multiple modes for the camera to move around in the three.js scene. You can set this mode with the poseMode parameter of your camera object. There are the following options:

  • ZapparThree.CameraPoseMode.Default: in this mode the camera stays at the origin of the scene, pointing down the negative Z axis. Any tracked anchors will move around in your scene as the user moves the physical camera and real-world tracked objects.
  • ZapparThree.CameraPoseMode.Attitude: the camera stays at the origin of the scene, but rotates as the user rotates the physical device. When the Zappar library initializes, the negative Z axis of world space points forward in front of the user.
  • ZapparThree.CameraPoseMode.AnchorOrigin: the origin of the scene is the center of the anchor specified by the camera's poseAnchorOrigin parameter. In this case the camera moves and rotates in world space around the anchor at the origin.

The correct choice of camera pose will depend on your given use case and content. Here are some examples you might like to consider when choosing which is best for you:

  • To have a light that always shines down from above the user, regardless of the angle of the device or anchors, use ZapparThree.CameraPoseMode.Attitude and place a light shining down the negative Y axis is world space.
  • In an application with a physics simulation of stacked blocks, and with gravity pointing down the negative Y axis of world space, using ZapparThree.CameraPoseMode.AnchorOrigin would allow the blocks to rest on a tracked image regardless of how the image is held by the user, while using ZapparThree.CameraPoseMode.Attitude would allow the user to tip the blocks off the image by tilting it.

With the camera set up, you can then create a tracked experience.

zapcode branded_zapcode i