CamTwist Effect Plugin to Convert THETA S Live Streaming Video from Fisheye to Equirectangular via Quartz

THETA Media
1

Kieran Farr put together a great start at streaming for the THETA S. This is a great way to learn how the conversion works.

For production use, you may want to look at the UVC Blender live streaming driver that RICOH released for Mac and Windows. Download the RICOH driver here.

A JavaScript alternative is thetaview.js

The source code for theta-s-quartz is available on GitHub.

An experiment to use OS X Quartz Composer to transform Ricoh Theta S live feed dual fisheye to equirectangular.

Goal is to create a Quartz composition that can be used in camtwist or other live video streaming software to adjust live the output of the theta-s to be suitable for using in a 360’ web panorama plugin. The flow will be something like this:

  • Theta S via USB or HDMI > Computer
  • Computer to use this Quartz composition from within Camtwist > Media encoder > HLS (via cloud encoding service) > web front-end
  • Web front-end needs equirectangular image, see this example: https://aframe.io/examples/showcase/sky/

Current state - Proof of concept works

  • theta-fish2sphere.qtz - this “routes” camtwist through Paul Bourke’s custom quartz patch project (see references below)

  • @kosso modified version added : theta-fish2sphere_K.qtz with reference images to play with.

Reference Input Image

![‘Reference Input Image’](https://github.com/kosso/theta-s-quartz/blob/master/THETAS_DUALFISH_REF_1280_720.png?raw=true =600x)

Test Output Snapshot

![‘Example Output Image … so far’](https://github.com/kosso/theta-s-quartz/blob/master/OUTPUT.png?raw=true =600x)

Example images:

To use:

  • Download camtwist http://camtwiststudio.com/, install to applications
  • Add theta-fish2sphere.qtz and theta-crop.qtz to Applications > CamTwist > Effects
  • Open CamTwist
  • Step 1: View > Main Window, Double click “Webcam” and choose RICOH THETA S
  • Step 2: Choose filter theta-fish2sphere
  • Step 3: Modify settings if you’d like, not sure it’ll make much of a difference yet. Make sure to press enter after you enter new values.

How to modify:

  • Download quartz composer from Apple’s developer site http://developer.apple.com/downloads by searching for “Graphics Tools for Xcode”
  • Open theta-fish2sphere.qtz file in quartz composer
  • Change input image source to theta s on USB or HDMI input live stream, or use sample image from this git project’s root dir

Dev notes for Quartz Composer

  • For testing / developing in QC, you’ll need to unhook the “Image (Required)” patch and use instead the image patch “image-example of 720p theta-s stream”
  • Ideally in QC during dev one could use video input patch to use as reference for modifying settings, but it is not consistent with behavior when ran inside of camtwist

Some helpful docs

Update

It is theoretically possible to experiment with the HDMI output of the Ricoh Theta S > Cable (Mini HDMI Output to Standard HDMI) > Capture Device (Blackmagic UltraStudio Mini HDMI Input to DisplayPort) > Quartz Custom Composition* > Quartz Syphon Plugin Output > OBS Syphon Receiver > RTMP > Streaming Service

*Quartz Custom Composition is the remaining step not complete. The current quartz composition in my git repo is only made for CamTwist and oriented for 720p not 1080p. It would be possible to adjust the Quartz Composition through (a) wise thinking with maths or (b) blind guessing and luck tweaking variables. (b) is my approach and it got us surprisingly close for 720p (UVC blender is still better).

Also caveat that *Quartz Custom Composition may require processing power above the capabilities of a standard MacBook Pro.

1 Like