Alpha Video

Studio

Alpha Video

ZapWorks Studio supports alpha video, video that features areas of transparency, using side-by-side masked MP4.

This article shows how to prepare and display alpha video using ZapWorks Studio and Premiere Pro. The video editing process should be similar in other tools.


Video Preparation

While this demonstration initially outputs a video with a resolution of 3840x1080 (from a 1920x1080 source file) we compress our final video to 1920x540 before importing it into ZapWorks Studio. Videos with a resolution higher than 1920 pixels in either dimension may not work on all devices.

The following steps arrange the color channels of the video on the left hand side of the frame, and the alpha channel on the right hand side.

We start with our video; here we have some footage of Zappar’s co-founder, Dr. Simon Taylor pretending to fly around like a familiar superhero who has jets on his feet and hands.

We imported this footage, which has a green screen background, into Premiere Pro and created a new sequence from it.

  1. Copy and paste this clip, and place the copy directly above the original in the timeline.

  1. Select the top clip and search for ‘Ultra Key’ in the Effects panel.

  1. Drag ‘Ultra Key’ onto the top clip and double-click to open the Effect Controls panel.

  1. Use the Eyedropper tool to select the Key Color - click on the green in the Program Monitor.

  1. Copy and paste the whole ‘Ultra Key’ effect onto the below clip.

  1. Reselect the top clip, and in the Ultra Key Effect Control, select ‘Alpha Channel’ as the Output.

Simon becomes a solid white shape.

  1. We need to make the sequence we have been working in twice as wide so select Sequence > Sequence Settings and change the Horizontal Frame Size to double the width of your original video (3840 in our case as the video was 1920 pixels wide).

  1. We need to position the clips so they are side by side. The position needs to be pixel perfect, so don’t move the clips by eye. Double click the top clip in the timeline to open the Effect Controls panel. Change the horizontal position to 75% of the width of the sequence (2880 in our case). The solid white version of Simon should move over to the right in the Program Monitor.

  1. Double click the bottom clip, and change the horizontal position to 25% of the width of the sequence (960 in our case).

The full-color version should move over to the left.

  1. Export the video as an h.264 mp4 file.

Below you can see the before and after of this process.


Before

After


  1. The final preparation step is to grab a single frame snapshot of the video for reference later on when creating our experience. In Premiere, you can create a snapshot by clicking the camera icon in the Program Monitor or using the appropriate keyboard shortcut (for us, on a Mac, it is Shift and E).

Snapshot


Studio Project (Embedded MP4)


  1. Start a new project and import your exported side-by-side MP4 video along with your snapshot image into the Media Library.

  1. Drag both the video and image into the Hierarchy.

  1. Select the snapshot image node in the Hierarchy and scroll down to the scale property within the Properties panel. Make a note of the first value (in our case 3.52) as it will be used in an upcoming step.

  1. Add two TransformedTextures which will allow us to separate out the left and right side of our video as if they were two separate video textures. We add TransformedTextures by right clicking on the root node and selecting 'New > Textures > TransformedTexture'. Do this twice to get two TransformedTexture nodes.

  1. Rename one of the TransformedTextures to "colorTexture" and the other to "maskTexture" to help differentiate them.

  1. Select the colorTexture and in the Properties panel set its texture to be that of the snapshot image. This will help us line things up for the final video texture which at this moment in time has no preview (that's why we are using the snapshot). After doing this, you should see the snapshot appear in a small window in the top right of the 3D view.

  1. We want to position and scale the colorTexture so that it fits perfectly on the color side of our video. To do this, we want to recall the scale number from before (3.52) and use that as the first (x-axis) scale value here. We then need to move the texture over to the color side of the snapshot, so we take half of the scale value (1.76) and use that to set the first (x-axis) position. With this done we can now see that the color portion of the video shown in the preview box in the top right of the 3D view.

  1. Now we want to setup our maskTexture in the same way following steps 6 and 7, however, this time we want the texture to cover the other side of the snapshot, so we set the first (x-axis) position value to -1.76 instead of 1.76. We now see the mask portion of the snapshot in the preview box.

  1. With the TransformedTextures setup using the snapshot for guidance, we can now switch both of their textures to the video. This will cause a default preview image to appear in the preview box, but the video will be used when the experience is run.

  1. Next, we want to create a new material that will use our TransformedTextures to create the alpha video effect. We do this by right-clicking the root node in Hierarchy and selecting 'New > Material' and naming it videoMaterial or something similar for clarification.

  1. Selecting the material node in the Hierarchy, we set the skin property to the color TransformedTexture, the mask property to the mask TransformedTexture, and the maskAlphaChannel to either r, g or b (we'll pick r). It doesn't matter because our mask is using black and white, so either of the color channels is fine.

  1. Now we need a plane to display our video on so we drag one from the Media Library into the Hierarchy renaming it "videoPlane".

  1. With our plane selected in the Hierarchy, we change its material property to the video material we created in step 10.

  1. We now just need to tell our video to play at some point during our experience, for ease of use we'll use the existing 'show' script. With the show script selected in the Hierarchy, we're presented with an editor view where we can drag our video node (not the video plane or material but our original video node) into. We can drag it in above the parent show function and insert it as a variable then allowing us to call the start function on it within the show function.

  1. The final thing we want to do before previewing our experience is to select the snapshot image we used for guidance earlier and set its visibility property to false, so this doesn't appear within our experience. We do this by selecting it in the Hierarchy and navigating to the visible property in the Properties panel and unchecking it.

  1. We can now preview our experience by selecting the preview option in the top left and giving the resulting code a scan with the Zappar app!

Download the completed project


  • It may take some fiddling around with the scale and positioning of the TransformedTextures to get them lined up correctly, it's helpful to click between the mask and color nodes to ensure they are lined up correctly.

  • The snapshot can be removed from the Hierarchy and Media Library once you have positioned the TransformedTextures as it's just used as a guide.


Streaming Alpha Video

The ZapWorks Studio streaming video implementation also supports alpha video.

This guide follows on from the embedded video project above and utilizes the video player symbol.


  1. Opening the project created above, create a video player symbol by clicking on the plus icon in the Symbol Definitions panel, selecting 'Video Player' and then clicking 'Create'.

  1. Drag the video player symbol from the Symbol Definitions panel into the Hierarchy beneath the videoPlane.

  1. Double click on the videoplayer symbol definition to enter it.

  1. Within the Hierarchy locate the backing plane within the controlsGroup group and untick the visible property in the properties panel with it selected.

  1. Do the same as the step above but this time for the bufferingBacking plane within the bufferingGroup group node.

And finally for the Plane0 plane.

The reason we are making these planes invisible is because we want to use the videoPlane we have in our parent symbol to display the video and not have the one's set up in the videoplayer symbol.


  1. Return to the parent symbol by double clicking the cog in the top left, just above the Hierarchy.

  1. Back in the parent symbol select the colorTexture node and set it's texture to none. Do the same for the maskTexture as well. We will set the texture in a script node in the following steps.

  1. Right click on the root node in the Hierarchy and select 'New > Script > blank' to create a new blank script.

Feel free to rename this script to something more suitable by right clicking the newly created script and selecting 'Rename'. This script will be used to set the texture of the transformed textures so we will call it 'textureAssignment'.

  1. With the script editor open click and drag the videoplayer node from the Hierarchy to the editor and select 'Insert Local Variable'. This will let us access the properties of the videoplayer, which we will need in following steps.

  1. Click and drag the root node into the script editor and select 'Add an event handler > show' to create an event handler function that executes when the show event is fired. This is where we will place our texture assignment code.

  1. Drag in both the color and mask texture nodes into the editor and insert them as local variables like we did with the videoplayer node.

  1. We can assign the textures to the transformed textures by typing out their variable names (the text before the = sign and after the var keyword), using the texture function (used for setting their texture) and then using some code to access the videoTexture from the video player symbol.

The code used to access the texture might be different depending on the name of your nodes but the default should be videoplayer0.nodes.show0.myvid.

With this code we are accessing the nodes within the video player symbol, getting the show node specifically, and asking for the myvid object, which is the texture in the video player symbol. It's not vital you understand everything that's going on in this step, just enough to make sure you are accessing the right component.

The complete code as shown above is as follows:

parent.on("show", () => {
    // Runs when show occurs on the parent node
    colorTexture.texture(videoplayer0.nodes.show0.myvid);
    maskTexture.texture(videoplayer0.nodes.show0.myvid);
});


  1. Before previewing the experience, select the videoplayer node and give it a URL to the video you wish to stream. For this example we can use the video we have here on the forum, the URL being:

https://d2j4z507ms5wl7.cloudfront.net/docs/simon-flying-masked.mp4

We can paste this URL into the videoURL property in the Properties panel. We can also set video to auto play and to show controls such as pause and play when tapping on the video. We do this by ticking the autoPlay and showControls checkboxes respectively.

  1. Preview the experience and give it a scan.

Download the completed project


When using your own video make sure that it is a publicly hosted MP4 or m3u8 (HLS Streaming) file hosted on a server that supports range requests.

  • If starting from scratch make sure you still grab a screenshot of the video for aligning the transformed textures.

  • While we used the video player symbol to help speed things up, you can use the underlying Z.Video technology for a more slimline solution.

Open in new window