Image Tracking

Image tracking can detect and track a flat image in 3D space. This is great for building content that's augmented onto business cards, posters, magazine pages, etc.

Before setting up image tracking, you must first add or replace any existing camera you have in your scene. Find out more here.

To track content from a flat image in the camera view, create a new ImageTracker object:

const imageTracker = new ZapparBabylon.ImageTracker();

Target File

ImageTrackers use a special 'target file' that's been generated from the source image you'd like to track. You can generate them using the ZapWorks command-line utility like this:

$ zapworks train myImage.png

The resulting file can be loaded into an image tracker object by passing it to the loadTarget(...) function as either a URL or an ArrayBuffer. The function returns a promise that resolves when the target file has been loaded successfully.

const imageTracker = new ZapparBabylon.ImageTracker();
imageTracker.loadTarget("myImage.zpt").then(() => {
    // Image target has been loaded
});

Alternatively the library provides a loader for loading a tracker and target file:

const imageTracker = new ZapparBabylon.ImageTrackerLoader().load("myImage.zpt");

Image Anchors

Each ImageTracker exposes anchors for images detected and tracked in the camera view. At this time, ImageTrackers only track one image in view at a time.

Anchors have the following parameters:

  • id: a string that's unique for this anchor
  • visible: a boolean indicating if this anchor is visible in the current camera frame
  • onVisible and onNotVisible: event handlers that emit when the anchor becomes visible, or disappears in the camera view. These events are emitted during your call to camera.updateFrame().

You can access the anchors of a tracker using its anchors parameter - it's a JavaScript Map keyed with the IDs of the anchors. Trackers will reuse existing non-visible anchors for new images that appear and thus, until ImageTracker supports tracking more than one image at a time, there is never more than one anchor managed by each ImageTracker. Each tracker also exposes a JavaScript Set of anchors visible in the current camera frame as its visible parameter.

To attach 3D content (e.g. Babylon.js objects or models) to an ImageTracker or an ImageAnchor, the library provides ImageAnchorTransformNode. It's a Babylon.js TransformNode that will follow the supplied anchor (or, in the case of a supplied ImageTracker, the anchor most recently visible in that tracker) in the 3D view:

const imageAnchorTransformNode = new ZapparBabylon.ImageAnchorTransformNode('tracker', camera, imageTracker, scene);

// Add in any 3D objects you'd like to track to this image
myModel.parent = imageAnchorTransformNode;

The TransformNode provides a coordinate system that has its origin at the center of the image, with positive X axis to the right, the positive Y axis towards the top and the positive Z axis coming up out of the plane of the image. The scale of the coordinate system is such that a Y value of +1 corresponds to the top of the image, and a Y value of -1 corresponds to the bottom of the image. The X axis positions of the left and right edges of the target image therefore depend on the aspect ratio of the image.

Events

In addition to using the anchors and visible parameters, ImageTrackers expose event handlers that you can use to be notified of changes in the anchors or their visibility. The events are emitted during your call to camera.updateFrame().

  • onNewAnchor - emitted when a new anchor is created by the tracker
  • onVisible - emitted when an anchor becomes visible in a camera frame
  • onNotVisible - emitted when an anchor goes from being visible in the previous camera frame, to being not visible in the current frame

Here's an example of using these events:

imageTracker.onNewAnchor.bind(anchor => {
    console.log("New anchor has appeared:", anchor.id);
    // You may like to create a new ImageAnchorTransformNode here for this anchor, and add it to your scene
});

imageTracker.onVisible.bind(anchor => {
    console.log("Anchor is visible:", anchor.id);
});

imageTracker.onNotVisible.bind(anchor => {
    console.log("Anchor is not visible:", anchor.id);
});
zapcode branded_zapcode i