Making Theta 360 work with OpenTok's client-side webRTC (livestreaming - mobile stitching - google "cardboard" view)

livestreaming
vr

#1

Hi there, just wanted to make a post on how discuss.io forked the theta_gl repository to enable 360 functionality in the desktop (firefox/chrome) and mobile (android only) browsers. I’ve spent the last month building this thing and wanted to thank everyone on the forum for letting me lurk and learn :slight_smile:

We recently presented it at conferences around the world and our customers are excited about using this technology to observe their customers in the real world!

You can find the PR here (pasted below): https://github.com/mganeko/THETA_GL/pull/2

Wanted to make a PR from what I’ve built to enable discuss.io to use the THETA 360 camera with OpenTok’s webRTC streaming service.

This PR adds new functions to call from the client side. Here are some examples:

var publisher360 = new _theta_gl();; // global pub/sub theta_gl instance

// to enable a 360 "preview" with OpenTok's publisher
var publisher = $('.OT_publisher').find('video')[0];
publisher360.init(/* divId */ 'self360', /* autoResize */ true,  /* debug */ false);
publisher360.overrideVideoElement(publisher);
publisher360.startAnimate();

// to disable a 360 "preview" over the OT stream
publisher360.stopAnimate();
$('#self360').find('canvas').remove();

There can be separate subscribers within a session. So you’ll need a different theta_gl instance for each one

var subStream = $( '#' + opentokID );
var subVideo; // defined on start360

if (subStream.theta_gl) {
  throw "subStream.theta_gl already exists. Cannot enable360subscriber more than once";
}
subStream.theta_gl = new _theta_gl();

// starting 360 on the subscriber element
if (!subVideo) {
    subVideo = subStream.find('video')[0];
}
subStream.theta_gl.init(/* divId */ 'sub360', /* autoResize */ true,  /* debug */ false);
subStream.theta_gl.overrideVideoElement(subVideo);
subStream.theta_gl.startAnimate();

// stopping 360 on the subscriber element
subStream.theta_gl.stopAnimate();
subStream.find('canvas').remove();

You can detect if it’s a mobile to device to enable/disable fullscreen

var mobile = /Mobi/.test(navigator.userAgent); // check to see if it's a mobile device

if (mobile) {
    var fullscreenCanvas = subStream.find('canvas')[0];
    subStream.theta_gl.disableStereoEffect();
    subStream.theta_gl.enableFullScreen(fullscreenCanvas, true);
}

Finally, you can use a UI element (with a high z-index) to toggle different modes. Here’s some example code that I use as an onclick function for these elements:

// the following code uses predefined start/stop 360Subscriber methods as exemplified in the previous code samples
    // add enable/disable functions to toggle elements
    function three60click(element, stereoView) {
      // change toggle, if canvas not present
      if (!subStream.find('canvas')[0]) {
        element.data('toggle', 'disabled');
      }

      // start/stop 360 subscriber tools
      switch (element.data('toggle')) {
        case "disabled":
          start360Subscriber();
          if (mobile) {
            subscribeElement.hide();
            cardboardMode.show();
            cardboardMode.data('toggle', 'enabled');
          }
          element.data('toggle', 'enabled');
          break;
        case "enabled":
          if (mobile) {
            if (stereoView) {
              subStream.theta_gl.enableStereoEffect();
              subStream.theta_gl.startAnimate();
              subscribeElement.show();
            } else {
              subStream.theta_gl.disableStereoEffect();
              // subStream.theta_gl.stopAnimate(); // need to fix half-screen
              subStream.theta_gl.startAnimate();
              subscribeElement.hide();
              cardboardMode.show();
            }
          } else {
            stop360Subscriber();
            element.data('toggle', 'disabled');
          }
          break;
        default:
          console.log('error');
        }
    }

Mapping THETA S dual-fisheye live streaming to a spherical surface in the browser
Trying to use the thetaV on windows in browser (chrome) for chatting and its not working
Mapping THETA S dual-fisheye live streaming to a spherical surface in the browser
Skype or video conferencing apps
#2

Nice work! Thank you for the contribution


#3

What service plan are you on with OpenTok? I’m just wondering if people need to pay $50/month minimum to experiment with this or if there is a pay-as-you-go or trial plan?


#4

I believe they start at $50 - they also have an enterprise plan that you have to contact them for. We’ve been debating whether or not to take the leap from the per-minute plans.

If you wanted to test things out, you should be able to contact them directly and ask for a trial API key.


#5

FYI for those following this thread. OpenTok updated its service plans to be a little more accessible for entry-level apps: https://tokbox.com/blog/build-with-the-opentok-platform-starting-at-9-99month

After developing with OpenTok for over 2 years now, I can say that it’s pretty impressive and takes a lot of the boilerplate, multi-browser support complexity out of the developer’s way.


#6

Thanks for the update. The $10/month plan is more accessible for people evaluating the feasibility of their application.

Have a nice day.