Drawing the Camera

This article directly follows on from Pipelines and Camera Processing, we highly recommend reading through that first.

The platform provides the most recent camera frame as a WebGL texture that you can draw to the screen. To make this texture available, call the following function after your frameUpdate() call:

pipeline.cameraFrameUploadGL();

Once uploaded, there are two ways to draw the camera:

  1. Using the cameraFrameDrawGL convenience function
  2. Manually drawing a full-screen quad with WebGL


Drawing the camera with cameraFrameDrawGL

You can use the following function:

pipeline.cameraFrameDrawGL(renderWidth: number, renderHeight: number, mirror?: boolean)

It will automatically draw the camera to the screen as a full screen quad. Please note this function modifies some GL state during its operation so you may need to reset the following GL state if you use it:

GL state Example
The currently bound texture 2D is set to null gl.bindTexture(gl.TEXTURE_2D, null)
The currently bound array buffer is set to null gl.bindBuffer(gl.ARRAY_BUFFER, null);
The currently bound program is set to null gl.useProgram(null)
These features are disabled: gl.SCISSOR_TEST, gl.DEPTH_TEST, gl.BLEND, gl.CULL_FACE

Manually drawing the camera

Alternatively, you can draw the camera manually to a full screen quad. To do so, you can use the following functions:

pipeline.cameraFrameTextureGL() : WebGLTexture | undefined: returns a WebGLTexture containing the current camera image (or undefined if none are available) pipeline.

cameraFrameTextureMatrix(renderWidth : number, renderHeight : number, mirror?: boolean) : Float32Array: pass in your canvas' width and height and it returns a 4x4 column-major matrix that you can use to transform the UV coordinates of the following quad:

Quad UV Coordinates
Vertex 0 -1, -1, 0
UV 0 -1, -1, 0
Vertex 1 -1, 1, 0
UV 1 0, 1
Vertex 2 1, -1, 0
UV 1 1, 0
Vertex 3 1, 1, 0
UV 1 1, 1

Here's an example vertex shader to show how this can be accomplished:

attribute vec4 position; // Bound to a buffer with the vertex data above
attribute vec4 texCoord; // Bound to a buffer with the UV data above
uniform mat4 texTransform; // Set to the matrix returned by cameraFrameTextureMatrix(...)

varying vec4 texVarying; // Used to pass the UV coordinates to the fragment shader

void main()
{
    gl_Position = position;
    texVarying = texTransform * texCoord;
}

And the corresponding fragment shader:

varying vec4 texVarying; // The UV coordinate from the vertex shader
uniform sampler2D texSampler; // Bound to the texture returned by cameraFrameTextureGL()

void main()
{
    gl_FragColor = texture2DProj(texSampler, texVarying);
}
zapcode branded_zapcode i