Unity WebXR implementation of camera.getLivePreview (XMLHttpRequest blocked by CORS)

THETA API Usage
, , , ,
1

Greetings everyone,

I am working with a Ricoh Theta V, firmware 3.60.1, focusing on wireless streaming with WiFi Client Mode.

I have found this Unity Project, updated by @KEI, in which a Monobehaviour script handles the MJPEG stream of camera.getLivePreview (Web API v2.1) utilizing the HttpWebRequest of System.Net (with or without Digest Auth).

I wanted to create an updated version of the project with WebGL and WebXR.
I will try to report most of the route I followed:

  • in order to use WebXR Export you need Unity Editor version 2019.4.7 and up or 2020.1 and up. So I end up using 2019.4.24f1
  • this how-to post explains pretty well building an unity project with WebXR except from the part where you must implement your own web server and not unity’s local Build and Run. I personally used Apache from XAMPP and created a Symbolic Link folder in Windows to easily export and run the WebGL project.

And now the issues part:

  • According to WebGL Networking you can’t use HttpWebRequest. WWW is obsolete so the only solution is UnityWebRequest, a class implemented from XMLHttpRequest.
  • UnityWebRequest doesn’t a have a Digest Auth feature as far as I searched into it. So I had to disable authentication in ThetaV with camera.setOptions and _authentication parameter via Postman. But be ware, I only succeeded in AP (blue led) WiFi mode.
  • When built and run from Chrome and Firefox (which are WebXR compatible) both browsers threw errors for Mixed Content (if you hit http://RICOH.THETA.IP.ADDRESS/osc/commands/execute from an https:// url). You can turn the error to a warning if you allow “Insecure Content” from browser=>settings=>site settings. Or simply just run WebGL project from http:// url (but the how-to post encourages https)
  • With that done, the CORS policy of the browser started complaining. You are most likely to come across this error:

Access to XMLHttpRequest at ‘http://RICOH.THETA.IP.ADDRESS/osc/commands/execute’ from origin ‘https://MY.PC.IP.ADDRESS’ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

I will not go into details about the CORS but it is built into browsers and difficultly surpassed. The 3 ways that I know of are:

  1. Make your requests to be “simple requests” in order to not trigger a preflight OPTIONS request and thus CORS policy.
  2. Moesif CORS and Allow CORS extentions for Chrome.
  3. Add some 'Access-Control-Allow-...' headers on the response of the server side (in this case the OSC server on Theta V).

As you can tell, for me doing this post I have not come to a solution.

  1. The POST /osc/commands/execute can not be a simple request because the Content-Type header must be 'application/json'.
  2. These extentions provide you the wanted Response-headers but a preflight OPTIONS request is still triggered. The Web API v2.1 conforms the OSC API which only uses GET and POST requests but not responding to OPTIONS requests (check also with Postman), resulting to the below error:

Access to XMLHttpRequest at ‘http://10.64.45.228/osc/commands/execute’ from origin ‘https://10.64.45.107’ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: It does not have HTTP ok status.

I didn’t commented on the 3rd listed solution, because I don’t know for sure that I can’t mess with the ThetaV HTTP server of the Web API running on port 80. If that is the case, I could maybe add functionality for responding to OPTIONS request and add the necessary response headers for CORS.

Another smart bypass solution that I see mainly on javascript applications is the cors-anywhere proxy or any custom proxy server that will have the functionality for browser’s CORS policy but also communicating with ThetaV for the MJPEG streaming. But I haven’t found a way to implement this embedded to my WebGL build.

In conclusion, I make this post hoping that some of you could help me with this.

Best regards,
Dimitris

2

There is source code for this project, but it requires a laptop to run on the code on. Is this not possible in your use?

Successful Theta V stream from drone to VR headset 0.25 miles away

Another approach with a Raspberry Pi as a relay is in this post

Industrial Robot Uses RICOH THETA 360° Live Video To Inspect Tight Places

Can you advise if it is possible to use another device as a relay in your application? Or, do you require the headset to connect directly with the camera?

3

The concept was to build an app fully wireless, portable and simple. That’s why I hoped that I could modify the HTTP server of ThetaV and anyone could indicate the right direction to do so.

I have seen and tested both amelia viewer and fox sewer rover (the rtsp version). The former uses native javascript and its an easy implementation with cors-anywhere module. The later is indeed a nice alternative to the functionality I want to achieve if I find a way to convert the equirectangular format to a 360-view in Three.js and use A-frame for VR camera.

Another possible solution, now that you mention a relay, is building my own proxy server on the Raspberry Pi 3 that provides ThetaV with WiFi.

4

I don’t know of a way to modify the HTTP headers that are coming from the THETA V built-in server for motionJPEG.

the underlying OS in the camera is Android. You can write a plug-in for the THETA V/Z1 that streams higher-quality video (like the RTSP plug-in) and have the relay basically be inside the camera.

It seems like you’ve researched this quite a bit.

The RTSP plug-in and the Wireless Streaming Plug-in are both open source. You can see what they did and try to modify it.

I’m assuming that the reverse proxy solution described below, won’t work or is not practical?

5

Thanks Craig, you are always helpful! I will reasearch further into the apache reverse proxy method, this is a possible solution. If I manage to come over this, I will write here again with an update.

1 Like