Publishing Custom Video Streams with the OpenTok API

Publishing Custom Video Streams with the OpenTok API

Published December 20, 2018 by Manik Sachdeva
Categories:

Growing up, I always wanted to be near the snow during the holiday season, but unfortunately, I live near San Francisco, so we never get snow. To make it feel like it’s snowing, let’s build a video chat app using the OpenTok API with moving snowflakes in our video stream!

Prerequisites

Before we get started, please make sure you have a TokBox account and the API Key and Secret from an API project.

Structure of the App

Create a directory and name it whatever you’d like:

We’ll go ahead and create a couple of files inside the directory using the following command:

Our project structure should now look like this:

Now that we’ve created our project let’s add some OpenTok code to our index.js file.

In the code above, we’ve initialized Session and Publisher objects using OT.initSession and OT.initPublisher methods,respectively. We then proceed to set event listeners on the session object for streamCreated and sessionConnected where we subscribe to a stream when it’s created and publish our stream when we’re connected to the session. After setting the session event listeners, we attempt to connect to the session using an OpenTok Token.

Let’s go ahead and add the index.js file to the index.html along with the OpenTok.js SDK.

Now that we know how to create a publisher and publish in a session, we need to create a custom video source using the Canvas API and the snowflake-greenscreen.mp4 video file.

Custom Video Source

In the code above, we use OT.getUserMedia, a wrapper of navigator.mediaDevices.getUserMedia, to get a MediaStream object. We then use the video MediaStreamTrack, from the MediaStream object to draw an image onto the canvas. After attaching the video MediaStreamTrack, we modify the image of the canvas by applying the snowflake filter.

Now that we’ve created a mechanism to capture the camera stream and add a snowflake filter to it let’s set the videoSource property for the publisher.

Note that in the code above, we used the captureStream method of the canvas object to get the resulting MediaStream object and call getVideoTracks()[0] on that object to get the video MediaStreamTrack object.

To start publishing your custom video in the session, add the apiKey, sessionId, and token values!

When you load your app, you should see the video from your camera with the moving snowflake filter:

Video stream with snowflakes

Known Limitations

  • The custom streaming API works on Chrome 51+, Firefox 49+ and Safari 11+. It does not work in IE or Edge browsers.
  • If the browser window loses focus—for example when you open a new tab—then the video will pause or become slow.

You can find all of the code for this sample application here. To see more samples of using OpenTok with Canvas, please visit the OpenTok-Web-Samples repo.

Leave a Reply

Your email address will not be published.