Solved: Unity Can't Display THETA V Live Stream on Windows 10

[craig]

You can test out the Lockheed Martin Amelia Drone app. View it in a browser and use A-Frame (included in the app) to set up the interface with the VR headset.

Otherwise, use the RTSP streaming method by @Hugues.

You can either run A-Frame in a browser or using a WebView (untested with the app) in your mobile project.

[anokhee]

Thank you.

The Amelia Viewer app did work for getting the live stream, with very less latency. However, we did not understand how A-Frame is used. I would appreciate if you could help us out with some guide or documentation to set up the interface with the VR headset/mobile phone.

I also wanted to ask how the VR headset/mobile phone is connected to the Amelia Viewer app. Is it a wired connection (through a USB cable) or is there some other way to connect them?

[craig]

There is extensive information here:

https://wiki.rit.edu/display/P203190/Amelia+Drone

RIT Amelia Drone Project (P19123)

[anokhee]

Thank you. The documentation was quite useful to understand how the interface with the VR headset is set.

However, we do not have an Oculus/Vive headset. So we are trying to look for a method which could help us do the same thing with any VR headset.

I would like to describe two tests that we conducted for the same.

Test 1: We tried the application that you had created for the project.

• We were able to connect and view the livestream from Theta V on the PC.
• We used Trinus VR software to view the livestream from the PC on the mobile device.
• However, the head tracking didn’t work.
• As we moved the phone around, the cursor on the PC moved, but the image didn’t. It is only when the cursor was clicked and dragged around, we could change the view of the image.
• I wanted to ask if there a workaround for that. We would want the view of the image to change as the phone is moved around.

Test 2: We tried to connect the phone by using the local network of the PC as a server. The source repository can be found here.

• The motion sensors could be accessed only over a secure connection, so we used a proxy server using https://

(The steps that we followed are enclosed here)

1. Download the source repo from here.

2. Open cmd.exe

3. Run: cd Downloads/amelia_viewer-master (Assuming the folder is in the Downloads folder)

4. Run: npm i

5. Run: npm start

6. ‘Ready for changes’ is displayed

7. The app opens up

8. Open another cmd.exe

9. Run: npm i -g https-proxy

10. Run: https-proxy --target=http://localhost:8181 --port=8182

11. ’ Proxing to “http://localhost:8181” through “htpps://localhost:8182” ’ is displayed

12. Open Firefox on the PC and go to http://127.0.0.1:8181/

13. Open the mobile browser

14. Open https://a.b.c.d:8182 (Replace it with your IP address)

• Head tracking worked perfectly here.
• However, when Theta V is connected to the PC, or when a local video is played on the PC, we could not view it on the mobile device.
• We could only see the first still image(the example) on the phone.
• We think that there is some part of the code that points to the example image and doesn’t change as the video is updated. However, we cannot understand what is causing this issue exactly.

There was an issue with the local video playback, which has also been attempted to be fixed in the source code that I have attached. But the loading timeout is 3000ms, so large videos could not be loaded. Increasing the timeout value also didn’t work for loading large videos. However, this is not a concern for us at the moment. I mentioned it because we were not sure if it is related to what is causing the issue with the livestream.

Could you please help us out here?

Thank you. Have a great day!

[Rob_Erta]

Hi everyone,
its 2020 and I feel like going back in time while not being able to get rid of the “Could not connect pins” error in unity.

I use Windows 10 with Unity 2020.1. I have updated the Ricoh Theta V to the latest firmware(3.50.1) and installed the current UVC blender driver.

Now I successfully managed to livestream to YouTube via OBS and also Unity recognizes the Theta.
However using the script attached to the sphere (as above and according to the unity documentation still up-to-date):

    Renderer renderer = this.GetComponentInChildren<Renderer>();
    WebCamTexture mycam = new WebCamTexture("RICOH THETA V/Z1 4K");
    renderer.material.mainTexture = mycam;
    mycam.Play();

gives me the error this whole thread started with:
“Could not connect pins - RenderStream()”

Anyone recently experienced the same? Is there any updated workaround?

[craig]

Are you using the updated driver? Lower right-hand corner of table. It is below the table.

https://support.theta360.com/en/download/

[dimdallas]

Hello @craig!

I have gone through the whole thread but I 've been experiencing the same problem as @Rob_Erta. I have installed the current Live-streaming app for Windows (RICOH THETA UVC V/Z1, v2.0.0).
OBS live-streaming works fine. Can there be a workaround?

[craig]

Hi, thanks for posting your question. It’s great to see Unity development still going strong.

I need to reinstall Windows 10 due to a hard disk problem and then I’ll run this again.

Can you post what version of the camera firmware, Unity, and Windows 10 you are using so that we have baseline information that might help trace the problem?

[dimdallas]

I am using Theta V with the latest firmware (3.50.1), Windows 10 Home edition and Unity Version 2019.4.6f1

[dimdallas]

Also @craig,
Would it be possible to give an answer to this comment of mine on Wireless streaming?

New Wireless Streaming Guide Available

Hello @codetricity! Is it possible that this wireless streaming approach with Theta V could work with a custom RTMP (like NGINX or MonaServer) rather than Youtube or Facebook? It would be even better if I could combine Wireless Live Streaming Plug-in with the THETA RTSP Streaming Plug-in (because I want least possible delay). Or the THETA RTSP Streaming Plug-in works wirelessly by default? Thank you in advance, Dimitris

Sorry for asking so many things at once,
DImitris

[pstawicki]

Hi @dimdallas,
I just tested some setup I was working on:
Windows 10 (20H2, build 19042.746) with WSL2 (windows subsystem for linux) with Ubuntu 18.04 LTS.
I installed NGINX (with rtmp module) on the Ubuntu (running in WSL2), example config, and on that same Windows machine watched the stream with ffmpeg/ffplay:
ffplay -fflags nobuffer -i "rtmp://192.168.2.118/live/z1"
z1 -> stream key.

The ffmpeg/fflpay works also for the “THETA RTSP Streaming Plug-In”.

Since this thread is about Unity, one could theoretically use FFMPEG as an plugin (dll) in Unity and project/pipe ffmpeg/ffplay output on a texture.
Piotr

[dimdallas]

@pstawicki

The ffmpeg/fflpay works also for the “THETA RTSP Streaming Plug-In”.

Hi @pstawicki,
I just tried the THETA RTSP Streaming Plug-In and I successfully viewed the stream with ffplay -fflags nobuffer -i rtsp://192.168.XXX.XXX:8554/live?resolution=1920x960

But I search for a solution for FFMPEG projection in Unity. Final target is either projecting on a inside-out sphere, or even better projecting on Skybox. I can’t find something for this problem, but I will keep on searching. If you have anything in mind, it will be usefull.

[craig]

See this project by @Hugues

Low latency (0.4s) H.264 livestreaming from Theta V WiFi to html5 clients with RTSP plug-in, ffmpeg and Janus Gateway on raspberry Pi

Hello, After many tries on Theta V, I achieved to get a H.264 livestreaming on a Raspberry Pi for multiples html5 clients with a 0.4s latency. It uses almost all realtime protocols available and powerfull softwares just to bypass USB livestreaming witch is not available on linux ! How it works: Theta V with RTSP plug-in connected in client mode (should work in AP mode too, 5GHz WiFi for better bandwith). RTSP with Device WebAPI Plug-in also works. ffmpeg (running on a Raspberry Pi) gets the…

it’s possible that the Janus WebRTC server is what you’re looking for.

https://janus.conf.meetecho.com/

the solution by @pstawicki using the RTMP module for NGINX looks like a great solution too.

[Rob_Erta]

Hi,
sorry for the huge delay. I was quite frustrated and put off. So indeed I used the link as described.
To be sure its not a different problem, I checked and confirmed that the YouTube LiveStream via OBSStudio works. So in principle my ThetaV allows live streaming, but it remains stubborn when it comes to Unity, even after adding DevicePath to the registry.

This is how the registry looks like:

Also I recorded a video of my workflow using the Theta in Live Video mode, maybe I am missing something … https://youtu.be/HVgttcTZ3Xw

I use the latest firmware on theta: 3.15.1
And Windows 10, 10.0.19042

[craig]

Uninstall the driver you are currently using and install the older driver. See the link below. We tested it with a V as well.

Can RICOH THETA Z1 be used in Unity as a webcam?

Hi everyone, We want to build a small VR conferencing app using Unity. I see that there were a lot of posts about connecting Ricoh Theta V with Unity via USB cable (like a webcam) to live stream 4K, but I couldn’t find any post about connecting the newer Z1 camera to Unity. Did someone successfully managed to connect it to Unity via USB? I like the larger sensor on Z1 and the image quality it produces, that is why I would like to go with it. Thank you and have a nice day.

RICOH THETA Z1 and V with Unity in 2021 for 360 Video Streaming

[Rob_Erta]

That was it! Big thanks for the video. Changes and development never stop. I am superhappy to start experimenting with Theta V in VR now.

[S_IWAKI]

Hello,
I have stuck with live streaming setting in Unity. I installed RICOH THETA UVC Driver 3.0.0 and tried live streaming. It works in OBS but not in Unity. Unity says “Couldn’t find a supported mediatype for the selected Virtual Camera”. I need a help!

Here is script that I’m using.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class ThetaV : MonoBehaviour
{
    void Start()
    {
        WebCamDevice[] devices = WebCamTexture.devices;
        Debug.Log("number of web cams connected:" + devices.Length);

        for (int i = 0; i < devices.Length; i++){
            string camName = devices[i].name;
            Debug.Log("webcam device "+ i +" is " + camName);
        }

        Renderer rend = this.GetComponentInChildren<Renderer>();

        WebCamTexture mycam = new WebCamTexture();
        string thetaName = devices[2].name;
        Debug.Log("The webcam name is " + thetaName);
        mycam.deviceName = thetaName;
        rend.material.mainTexture = mycam;

        mycam.Play();
    }

    // Update is called once per frame
    void Update()
    {

    }
}

Devices:
RICOH THETA V
Firmware 3.82.1
RICOH THETA UVC Driver 3.0.0
Unity 2022.3.0

Sorry for my weird English. I’m not native speaker.

[jcasman]

@S_IWAKI

It works in OBS but not in Unity.

@S_IWAKI Have you tried posting in the Unity forum? Since this sounds like (possibly) a Unity problem, you may get help there.

I have posted here in the past, the AR/VR (XR) Discussion part of the Unity forum: https://forum.unity.com/forums/ar-vr-xr-discussion.80/