Camera Setup

The Zappar Babylon library provides a camera object that you can use instead of Babylon's Camera. It is constructed using the following function:

const camera = new ZapparBabylon.Camera('camera', scene);

Options

Options may be passed into the ZapparBabylon.Camera constructor.

Custom Video Devices

const camera = new ZapparBabylon.Camera(
    "camera",
    scene,
    {
        rearCameraSource:'csO9c0YpAf274OuCPUA53CNE0YHlIr2yXCi+SqfBZZ8=',
        userCameraSource: 'RKxXByjnabbADGQNNZqLVLdmXlS0YkETYCIbg+XxnvM='
    }
);

Find out more information here.

Clipping Planes

const camera = new ZapparBabylon.Camera(
    "camera",
    scene,
    { // These values are defaults.
        zNear: 0.1,
        zFar: 100
    }
);

Processing Camera Frames

Call the following function once an animation frame (for example during your engine.runRenderLoop function) in order to process incoming camera frames:

camera.updateFrame();

Alternatively, you may allow the camera to update itself (no longer requiring the step above) by constructing it as so:

const camera = new ZapparBabylon.Camera('camera', scene, true); // Self updating camera

Permissions

The library needs to ask the user for permission to access the camera and motion sensors on the device.

To do this, you can use the following function to show a built-in UI informing the user of the need and providing a button to trigger the browser's permission prompts. The function returns a promise that lets you know if the user granted the permissions or not.

// Show Zappar's built-in UI to request camera permissions
ZapparBabylon.permissionRequestUI().then(granted => {
    if (granted) {
        // User granted the permissions so start the camera
        camera.start();
    } else {
        // User denied the permissions so show Zappar's built-in 'permission denied' UI
        ZapparBabylon.permissionDeniedUI();
    }
});

If you'd rather show your own permissions UI, you can use the following function to trigger the browser's permission prompts directly. The function returns a promise that resolves to true if the user granted all the necessary permissions, otherwise false. Please note - due to browser restrictions, this function must be called from within a user event, e.g. in the event handler of a button click.

ZapparBabylon.permissionRequest().then(granted => {
    if (granted) {
        // User granted the permissions so start the camera
    } else {
        // User denied the permissions
        // You can show your own 'permission denied' UI here or use Zappar's built-in one
        ZapparBabylon.permissionDeniedUI();
    }
});

Starting the Camera

Once the user has granted the necessary permissions, you can start the rear-facing camera on the device with the following function on your camera object:

camera.start();

If you'd like to start the user-facing 'selfie' camera, pass true:

camera.start(true);

To switch between the front- and rear-facing cameras during your experience, just call camera.start(...) again passing either true or false as appropriate.

User-facing cameras are normally shown mirrored to users and so the camera object provides a number of modes to support this:

ZapparBabylon.CameraMirrorMode.Poses: This mode mirrors the background camera texture and ensures content still appears correctly tracked. In this mode your content itself isn't flipped, so any text in your tracked content doesn't appear mirrored. This is the default mode for the user-facing camera.
ZapparBabylon.CameraMirrorMode.CSS: In this mode, the Zappar camera applies a scaleX(-1) CSS transform to your whole canvas. This way both the camera and your content appear mirrored.
ZapparBabylon.CameraMirrorMode.None: Mo mirroring is performed. This is the default mode for the rear-facing camera. The userCameraMirrorMode and rearCameraMirrorMode parameters set which mode the camera object will use for each camera:

camera.userCameraMirrorMode = ZapparBabylon.CameraMirrorMode.CSS;

Realtime Camera-based Reflections

The SDK provides an automatically generated environment map that's useful if you're using materials that support reflections (e.g. PBRMetallicRoughnessMaterial, StandardMaterial). The map uses the camera feed to create an approximate environment that can add some realism to your scene.

To use the map, first construct an instance:

const envMap = new ZapparBabylon.CameraEnvironmentMap(camera, engine);

Then, attach the map to specific materials:

material.environmentTexture = envMap.environmentMap;

Finally, call update() on the map each frame, between updating the camera and rendering the scene:

engine.runRenderLoop(() => {
  envMap.update();
  scene.render();
});

Camera Pose

The Zappar library provides multiple modes for the camera to move around in the Babylon.js scene. You can set this mode with the poseMode parameter of your camera object. There are the following options:

ZapparBabylon.CameraPoseMode.Default: In this mode the camera stays at the origin of the scene, pointing down the negative Z axis. Any tracked anchors will move around in your scene as the user moves the physical camera and real-world tracked objects.
ZapparBabylon.CameraPoseMode.Attitude: the camera stays at the origin of the scene, but rotates as the user rotates the physical device. When the Zappar library initializes, the negative Z axis of world space points forward in front of the user.
ZapparBabylon.CameraPoseMode.AnchorOrigin: the origin of the scene is the center of the anchor specified by the camera's poseAnchorOrigin parameter. In this case the camera moves and rotates in world space around the anchor at the origin.

The correct choice of camera pose with depend on your given use case and content. Here are some examples you might like to consider when choosing which is best for you:

To have a light that always shines down from above the user, regardless of the angle of the device or anchors, use ZapparBabylon.CameraPoseMode.Attitude and place a light shining down the negative Y axis is world space.
In an application with a physics simulation of stacked blocks, and with gravity pointing down the negative Y axis of world space, using ZapparBabylon.CameraPoseMode.AnchorOrigin would allow the blocks to rest on a tracked image regardless of how the image is held by the user, while using ZapparBabylon.CameraPoseMode.Attitude would allow the user to tip the blocks off the image by tilting it.

With the camera set up, you can then create a tracked experience.