Can RICOH THETA Z1 be used in Unity as a webcam?

[octav1an]

Hi everyone,
We want to build a small VR conferencing app using Unity. I see that there were a lot of posts about connecting Ricoh Theta V with Unity via USB cable (like a webcam) to live stream 4K, but I couldn’t find any post about connecting the newer Z1 camera to Unity. Did someone successfully managed to connect it to Unity via USB? I like the larger sensor on Z1 and the image quality it produces, that is why I would like to go with it.

Thank you and have a nice day.

[RicoB]

Hey ctav1an,

Just curious if you’ve had luck streaming the camera directly in Unity? I keep getting “Could not connect pins - RenderStream()” when playing the scene.

The only way I’ve managed to do it is with a ridiculous hack, just so I can at least develop while finding a real solution:

• Plug Ricoh Z1 by USB
• Open stream in MPC-HC media player
• Capture with OBS
• Output to Unity wit NDI
• Receive in Unity with NDI receiver
• Apply to Unity Skybox.

I mean…it works haha. But definitely not a realistic or usable option.

[octav1an]

We ended up buying the RICOH Theta V, and it works fine with Unity. If you manage to solve the problem could you please make a post here. Thanks.

[craig]

Thank you for the update. Sorry you had to buy a THETA V. I haven’t got around to testing it with the Z1, but still intend to.

it appears that @RicoB has got it to work with a hack, likely the hack described below.

The only way I’ve managed to do it is with a ridiculous hack, just so I can at least develop while finding a real solution:

You can likely edit the register with regedit and create a fake device driver for RICOH THETA Z1 4K. I have not tried this yet. You can find the HKEY by searching for RICOH THETA

Solved: Unity Can't Display THETA V Live Stream on Windows 10

Great! I guess that must be the solution! Man now I feel guilty for forgetting to post about it a while ago. Anyway I’ll go ahead and recap the solution for this thread. Open regedit Navigate to: For RICOH THETA V FullHD: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID{860BB310-5D01-11D0-BD3B-00A0C911CE86}\Instance{05B5709A-D045-4076-AA59-CAF159FEC26E} For RICOH THETA V 4K: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID{860BB310-5D01-11D0-BD3B-00A0C911CE86}\Instance{44ADD235-FE8A-4…

This is for the THETA V.

See this article

Search for THETA Z1 and find the key (have not tried yet)

The key will be different for Z1.

From roughly Mar 2018, the hack was no longer needed with THETA V as RICOH incorporated a similar hack into the driver 1.0.1.

What I want to do is to first test this myself with the Z1 and then send a report to some guys I know at RICOH.

If you hack the registry on your Windows machine, you will need to hack the register for every Windows computer you use with the Z1. Thus, it’s not a good solution for commercial deployment if you sell your app. However, it may be useful for prototyping and demos. Also, I don’t know if it works with Z1 as I have not tested it yet.

[RicoB]

Hey @craig

Btw, the hack I did was meant as a figure of speech hack…not a real “registry hack”. I just streamed the ricoh camera from MPC-HC to OBS, through NDI, to Unity. It was just for testing purposes. Quality and delay was pretty bad.

So I created a new Key under “Instance” using a CLSID I found for the Z1 searching the registry (It seems the key name is the same as the CLSID for all other cameras, so I did that), then added all the necessary String inside (ie. FriendlyName, DevicePath, CLID,etc. - except FilterData cause I couldn’t find it or understand). Results: same issue, unfortunately.

Here are my findings, though:

• The new key I created makes it so MPC-HC now sees 2 x Ricoh Z1, but the new one doesn’t work.
• I found the registry entry that MPC-HC uses to find the camera, by doing a search in the registry and changing the “FriendlyName” until I saw it changed it in MPC-HC as well. I added a “DevicePath” entry for the fun of it, but that didn’t do anything. Here’s the path in case anyone else wants to experiment:

HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\DeviceClasses{65e8773d-8f56-11d0-a3b9-00a0c9223196}##?#USB#VID_05CA&PID_2715&MI_01#7&b5856d3&0&0001#{65e8773d-8f56-11d0-a3b9-00a0c9223196}#GLOBAL\Device Parameters

• In Unity, if the Z1 is in live mode, I am getting a black texture on the sphere/object I am applying the texture to, and it is saying my current camera is the Z1 in the debug window, but it still has that “Could not connect pins - RenderStream()” error right after.
• If I unplug the camera, the sphere displays a texture that just shows the Ricoh Theta logo with “Theta V FullHD Status: 0x800705AA” error message. So it just finds the next available camera to use as a texture, which I don’t actually have.

[codetricity]

@RicoB, thank you for reporting back with these detailed results. I will take two actions in parallel.

1. I will report this back to some people I know at RICOH
2. I will also attempt to either replicate the error or get the “fake driver” working

Your information is useful. It sounds similar to the problem we had with the THETA V initially. If you have time, can you post the version of Unity you are using.

[RicoB]

That’s great news! It’s nice to see a forum being so active and people interested in resolving issues for a relatively “niche” product. Or maybe I hadn’t realized how big the Ricoh 360 camera customer base had grown in the past few years.

Anyway, all my tests have been with Unity 2019.1.0f2. I will try a blank project with the newest version of unity on Monday, though. Was delaying upgrading since we already have a working project using our Nokia OZO, and it’s always a risk upgrading with a product that is no longer technically supported.

[RicoB]

Alright, so I tried with the newest version of Unity today and it still has the same issue.

One thing I noticed is when I stream the camera live in MPC-HC, a big “LIVE” word starts blinking on the camera’s small screen. When I play the scene in Unity, the camera quickly flashes the “LIVE” words then it disappears, as if it connected to it for a second, then just disconnects due to the “connect pins” issue.

[craig]

Thanks for this report on your tests. We saw a similar thing with the THETA V. MPC-HC could detect the hardware device driver (THETA V or THETA Z1) on Windows. However, other software such as Unity and OBS (Open Broadcaster Software) required the use of the software device driver. In addition, Unity also needed non-blank fields as described above.

Just so you know, for the THETA V,

Web Cam Device	Used On	Note
THETA V	Mac OS and Windows only with MPC_HC	no driver software needed
THETA V FullHD and THETA V 4K	Other Windows software like OBS	driver software needed on Windows
THETA V FullHD and THETA V 4K with update for Unity	Works on most Windows software, including Unity	Basically incorporated the community hack for “software driver info” into the official driver

The Windows software for the THETA V I’m referring to is here:

https://topics.theta360.com/en/faq/c_06_v/304_1/

This software may need to be updated for the THETA Z1.

I’m sharing old information right now, I need to test this more with the Z1 before I have an opinion on the current state of Z1 live streaming with Unity.

Note that if you have access to MacOS, it might be worth trying Unity on MacOS as it uses a different driver. The test below with the THETA V was done on a Windows machine with dual-boot MacOS.

HowTo: Unity and THETA V Live Streaming - Mac OS X

The THETA V streams a 4K equirectangular video over USB. It appears to Unity as a normal webcam called RICOH THETA V. The video is already stitched. You just need to attach a WebCamTexture to a 3D sphere and you’ll have a simple VR application. To get started with live streaming an equirectangular video from the THETA V, first create a new project in Unity. Add A Sphere [image] Scale the sphere to a reasonable size such as 8 8 8. [image] You’ll have a simple scene like this. It just has a…

Update 10/1 I am looking for the software driver for the Z1. Have you seen any download links for a Z1 live streaming driver? I have a note out to a person at RICOH about this.

On the Z1, I have this device on Windows 10, but it is not appearing in OBS.

If it’s not working on OBS, I don’t think it will work on Unity. I’m going to try and get the Z1 to stream with OBS before I test it with Unity.

I tried to stream with MPC-HC and I can’t get the video stream to appear right now.

[craig]

@RicoB

One thing I noticed is when I stream the camera live in MPC-HC, a big “LIVE” word starts blinking on the camera’s small screen.

Is the video stream appearing inside of the preview window of MPC-HC? Can you post a screenshot of your settings?

[RicoB]

@craig I finally got the Z1 working in Unity and OBS! It’s probably because of you, since you said you might be able to contact people at Ricoh.

They just released a UVC4K software for the Z1. Here’s the link (found it in the FAQ): Live video is not displayed on a Computer - トピックス(米国)

Once installed, it created a new “Camera” entry in the registry called “RICOH THETA Z1 FullHD” (Probably does other magic too, not sure). But now, you can specify that name instead of the regular “RICOH THETA Z1” in your code in Unity, it works. In OBS, “RICOH THETA Z1 FullHD” now appears as a camera choice.

Anyway, thanks so much for your help @craig

Here’s my code that I attached to a sphere to test it:

using UnityEngine;

public class getRicohStream : MonoBehaviour
{

static WebCamTexture myCam;
string selectedDeviceName = "RICOH THETA Z1 FullHD";

void Start()
{

    Renderer renderer = GetComponent<Renderer>();

    if (myCam == null)
            myCam = new WebCamTexture(selectedDeviceName, 1920, 960);

        renderer.material.mainTexture = myCam;

    if (!myCam.isPlaying)
    myCam.Play();

}

}

[craig]

Nice! This is fantastic. I have it working with OBS and am upgrading Unity and Visual Studio in order to change my application from THETA V 4K to THETA Z1 4K. Just an FYI, the 4K driver is higher resolution, but you may experience lag if you’re transmitting the stream over the Internet.

[RicoB]

Great! I could only get FullHD version to be recognized in Unity for some reason. It is showing up in the registry, so I’m not sure why. I’ll try reinstalling it, or maybe even restarting my computer

By the way, I posted a little guide for anyone interested in getting their camera stream as a Unity skybox instead of a sphere:

Stream Ricoh Theta to Unity Skybox

Note: This was done with the Theta Z1, but should work with the others too. Thought I’d share this here to help improve the quality of projects people are working on. I noticed a lot of people are projecting the cam texture on a “inverted” sphere. The problem is that sphere’s are still polygons and create “edges” in your 360 image. It’s not pretty or professional. Even worse is if you decide to use it in room-scale VR - the sphere won’t move with you and it’ll just be weird (unless you make th…

[derf]

Thank You for this! That conversation was exactly what I was going through. I got the 4K in Unity as well though following the same steps you did.

[Yanjiao_Ao]

@RicoB

Hi! Thank you for the information. I am facing the same problem now. I found that the current drivers Ricoh theta v/z1 4k and Ricoh theta v/z1 FullHD work perfectly in OBS. But in Unity, it constantly shows the Could not connect pins - RenderStream() error. May I ask which version of Unity you have tested with?

[dimdallas]

Hello @RicoB,

I also happen to have the same problem with Could not connect pins - RenderStream() error. As @Yanjiao_Ao asked, mine version of Unity is 2019.4.6f1 and running from Windows 10 Home edition. I have gone through this topic with the same error from @codetricity , but it hasn’t been solved.

Has there been any aditional work you did with this error that you could share? Thanks

[craig]

Please confirm that you are using the driver from here:

https://topics.theta360.com/en/faq/c_00_z1/304_1/

If so, please confirm:

• you are using Windows 10 64 bit
• post build number of Windows
• post camera firmware version
• confirm that the “live” word is flashing on the front of the camera (you most push the mode button a few times to see it)

[dimdallas]

Hey @craig!

Just to clarify, I talk about THETA V.
I confirm that I unistalled RICOH THETA UVC V/Z1 v2.0.0 and made a fresh install from the link here

I also confirm the info you asked:

• I use Windows 10 Home 64-bit
• Version 2004 (OS Build 19041.804) as provided by “winver” command
• camera firmware version 3.50.1
• Live led flashes and in addition streams runs perfectly in OBS (video and audio)

[craig]

I have the same problem.

I’m using a Skybox.
Unity 2019.4.20f1

I’m going to take another look at this tomorrow.

[Justin]

So I had this same issue with the Ricoh Theta V and just recently found a band-aid solution that seems to work for now until someone finds something better.
I looked around and started running through various Unity versions even ones that had been previously confirmed to work but they all gave that same pins error. I finally tried uninstalling the recommended UVC drivers that have been linked above and instead installing older drivers that @craig inked in this post:

Solved: Unity Can't Display THETA V Live Stream on Windows 10

UPDATE - Hack No Longer Needed RICOH THETA UVC 4K 1.0.1 appears to solve this problem. Download New Driver version 1.0.1 https://theta360.com/en/support/faq/c_06_v/304_1/ download 64 bit for Windows download 32 bit for Windows Uninstall Previous Driver [image] Install New Driver [image] Reboot computer. Test Unity [image] It works! Use Regedit to Confirm That Hacked DevicePath is Not Used I used foo:bar in the hack. New DevicePath set by driver is RICOH THETA V. It appears that…

The drivers specifically:
Windows 64-bit: https://theta360.com/en/support/download/liveapp4k/win64
Windows 32-bit: https://theta360.com/en/support/download/liveapp4k/win32

If you do this make sure to uninstall the recommended drivers or it won’t work. Then in Unity instead of trying to read the Ricoh Theta V/Z1 FullHD or Ricoh Theta V/Z1 4k streams, you will read the Ricoh Theta V FullHD or Ricoh Theta V 4k streams.

My current setup:
Using the skybox method
Unity 2020.2.4f1 (though I think I tested it on earlier versions and it also works)
Ricoh Theta V
Firmware Version: 3.50.1
Microsoft Windows 10 Education

(First time posting here so let me know if there is anything else that I should add)