Solved: Unity Can't Display THETA V Live Stream on Windows 10


#33

Thanks for the update.

Using Blender does work, but I understand that it disrupts the workflow if you’re building different spheres or want to change vertices. It’d also be cool and educational to do this entirely in Unity.

The code below is from Shanyan Teng. It was made for a video file, but it may be possible to modify it for the live stream. Unfortunately, I don’t understand it right now, though I’ve tested it with video file applications and it does work.

I’ll carve out some time and try it myself in the future.

Shader "ThetaInsideShader" {
	Properties{
		_MainTex("Base (RGB)", 2D) = "white" {}
	}

		SubShader{
		Tags{ "RenderType" = "Opaque" }
		Cull front    //  FLIP THE SURFACES
		LOD 100

		Pass{
		CGPROGRAM
#pragma vertex vert
#pragma fragment frag

#include "UnityCG.cginc"

		struct appdata_t {
		float4 vertex : POSITION;
		float2 texcoord : TEXCOORD0;
	};

	struct v2f {
		float4 vertex : SV_POSITION;
		half2 texcoord : TEXCOORD0;
	};

	sampler2D _MainTex;
	float4 _MainTex_ST;

	v2f vert(appdata_t v)
	{
		v2f o;
		o.vertex = mul(UNITY_MATRIX_MVP, v.vertex);
		v.texcoord.x = 1 - v.texcoord.x;
		o.texcoord = TRANSFORM_TEX(v.texcoord, _MainTex);
		return o;
	}

	fixed4 frag(v2f i) : SV_Target
	{
		fixed4 col = tex2D(_MainTex, i.texcoord);
	return col;
	}
		ENDCG
	}
	}

}

#34

Updated test of 4K streaming on Windows in Unity. Now showing view from inside of Sphere.

Here’s the Unity package used in the test.

thetav-streaming-test.unitypackage (656.5 KB)

Other tests:

  • Did not work on Unity 2017. I’m still using Unity 5.5
  • Could not get code above to work with a normal Unity sphere. I’m using a Blender ico sphere with flip normals

#35

@joshappleman Thanks! This fix worked for my purposes! Would be nice to see some other fixes without use of the blender sphere at some point though :slight_smile:


#36

Just so everyone following this thread is clear, the fix Megan is talking about is to set the x, y, z scale values to negative numbers. Otherwise, the text and other things will appear reversed. This is not implemented in the unitypackage file I shared above. I created another package below with the blender sphere and with the default scale set to -8, -8, -8, so it shows the correct mirroring from the start.

Also, there should be examples of using Unity to project the stream onto the inside of a sphere without using Blender. We should look for an example and publish the results here.

Now with negative values for scale of x, y, z, per @joshappleman’s tip. :slight_smile:

Shows newspaper headline to verify that mirroring problem is resolved.

thetav-flip-normals-sphere.unitypackage (656.5 KB)


#37

THETA V Now Working with Unity 2017.2.0f3

Process

There’s another virtual driver that needs to be edited.

In Regedit, use the find command to locate all “Data” for THETA V.
image

Once you find the key with the search feature, click on each Instance.

image

Prior to editing, you will not see a DevicePath.

image

You need to add it.

Right-click on the Instance and select New->String Value.
image

Add DevicePath. Right click on DevicePath again and then modify it. Add a fake path like foo:bar.

image

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Classes\WOW6432Node\CLSID\{860BB310-5D01-11d0-BD3B-00A0C911CE86}\Instance\{D36BCC1B-8993-4568-B265-DC5AE05A8C8B}

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Classes\WOW6432Node\CLSID\{860BB310-5D01-11d0-BD3B-00A0C911CE86}\Instance\{C0A73D6F-A72E-4064-B648-BDCB30C09E6A}


image

Summary

NEW Info for THETA V 4K

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Classes\WOW6432Node\CLSID{860BB310-5D01-11d0-BD3B-00A0C911CE86}\Instance{D36BCC1B-8993-4568-B265-DC5AE05A8C8B}

Previous fix was limited to this.

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID{860BB310-5D01-11d0-BD3B-00A0C911CE86}\Instance{44ADD235-FE8A-42D8-8B18-CF575554AAB1}


#39

UPDATE - Hack No Longer Needed

RICOH THETA UVC 4K 1.0.1 appears to solve this problem.

Download New Driver version 1.0.1

https://theta360.com/en/support/faq/c_06_v/304_1/

Uninstall Previous Driver

image

Install New Driver

image

Reboot computer.

Test Unity

It works!

Use Regedit to Confirm That Hacked DevicePath is Not Used

I used foo:bar in the hack. New DevicePath set by driver is RICOH THETA V. It appears that Unity developers no longer need to use regedit.

image

Thanks Ricoh!

It appears that Ricoh listened to the community hack and implemented a DevicePath into the driver that people can download from the Ricoh site.


HowTo: Use RICOH THETA V with HTC Vive and SteamVR
#40

Hi I have the same problem with my theta V and Window 10, could you upload the project?


#41

I think your problem will be solved if you download the latest RICOH THETA UVC 4K 1.0.1

https://theta360.com/en/support/faq/c_06_v/304_1/

https://theta360.com/en/support/download/liveapp4k/win64


A simple test package is available a few posts above. Though, the problem is probably the driver if your problem is that Unity can’t use the THETA V as a web cam.

thetav-flip-normals-sphere.unitypackage (656.5 KB)

If your problem is that the image is mirrored, the above package may help. By mirrored, I mean that the text on a newspaper looks like it does in a mirror. If that is your problem, than the package above will help.


Trying to use the thetaV on windows in browser (chrome) for chatting and its not working
#42

Thank you! It worked! :wink:


#43

Awesome. Would love to hear about your project if you have a moment.

Right now, I’m trying to figure out the proper height for the THETA V to give a good telepresence experience. I think I’m going to need to adjust the virtual camera height in Unity as well.


#44

telepresence experience sounds very interesting. Maybe it fits well with the idea I am working on.
I am working on a physical sphere with ThetaV inside. Like a inner body experience.
The user can touch the sphere and see him/herself from inside.

Now I am trying to add sound. Do you know how to use the Audio from Theta V?


#45

Hello, I did it by adding the following code in my basic script:

addingSound

I hope it helps you,

N


#46

thank’s! I am quite new in Unity 3D, so my question is what you mean with basic script?


#47

I am just meaning the one attached to the sphere.
It is the same script where you catch the texture from the Theta V.

N


#48

@Litto
Create the script in C# (or language of choice) and then attach it to the virtual sphere. You can drag and drop it.

Here’s a basic script that will recognize the camera. I have not implemented audio yet per the @Nima suggestion. I will try it today. If you successfully add audio to the script below, post the full working example. Thanks.

using UnityEngine;
using System.Collections;

public class webCamDetect : MonoBehaviour
{
	string camName;
	public const string RICOH_DRIVER_NAME = "RICOH THETA V 4K";
	// change to "RICOH THETA V FullHD" for lower resolution 
	// (and thus smaller data size)

	void Start()
	{
		WebCamDevice[] devices = WebCamTexture.devices;
		Debug.Log("Number of web cams connected: " + devices.Length);
		for (int i = 0; i < devices.Length; i++)
		{
			Debug.Log(i + " " + devices[i].name);
			if (devices[i].name ==  RICOH_DRIVER_NAME)
			{
				camName = devices[i].name;
			}
		}

		Debug.Log("I am using the webcam named " + camName);

		if (camName != RICOH_DRIVER_NAME)
		{
			Debug.Log("ERROR: " + RICOH_DRIVER_NAME +  
				" not found. Install Ricoh UVC driver 1.0.1 or higher. Make sure your camera is in live streaming mode");
		}

		Renderer rend = this.GetComponentInChildren<Renderer>();
		WebCamTexture mycam = new WebCamTexture();

		mycam.deviceName = camName;
		rend.material.mainTexture = mycam;

		mycam.Play();
	}
}

#49

Is the spatial audio input from the THETA V working with the HTC Vive or Oculus Rift (or other) headset? Is it 4 channel audio or 2 channel?


#50

I tried your script - I get an error
Assets/testScriptBasic.cs(40,10): error CS1525: Unexpected symbol `end-of-file’

I used this script, which worked fine (without audio):

using UnityEngine;
using System.Collections;

public class testScriptBasic : MonoBehaviour {

void Start() {
    WebCamDevice[] devices = WebCamTexture.devices;
    Debug.Log("Number of web cams connected: " + devices.Length);
    Renderer rend = this.GetComponentInChildren<Renderer>();

    WebCamTexture mycam = new WebCamTexture();
    string camName = devices[2].name;
    Debug.Log("The webcam name is " + camName);
    mycam.deviceName = camName;
    rend.material.mainTexture = mycam;

    mycam.Play();
    }
}

#51

Update on Spatial Audio Tests

to followup on the discussion between @nima and @Litto

  • I have the THETA working as a spatial audio microphone inside of Unity
  • I can play spatial audio on headphones attached to my computer
  • I can get sound inside of the HTC Vive headset
  • I have not done precise tests of the orientation of the spatial sound inside the headset

Script for THETA V Live Streaming plus basic audio

using UnityEngine;
using System.Collections;

public class webCamDetect : MonoBehaviour
{
	string camName;
	public const string RICOH_DRIVER_NAME = "RICOH THETA V 4K";
	// change to "RICOH THETA V FullHD" for lower resolution 
	// (and thus smaller data size)

	// Audio
	public const int THETA_V_AUDIO_NUMBER = 0;   
	AudioSource audioSource;

	void Start()
	{
		WebCamDevice[] devices = WebCamTexture.devices;
		Debug.Log("Number of web cams connected: " + devices.Length);
		for (int i = 0; i < devices.Length; i++)
		{
			Debug.Log(i + " " + devices[i].name);
			if (devices[i].name == RICOH_DRIVER_NAME)
			{
				camName = devices[i].name;
			}
		}

		Debug.Log("I am using the webcam named " + camName);

		if (camName != RICOH_DRIVER_NAME)
		{
			Debug.Log("ERROR: " + RICOH_DRIVER_NAME +
				" not found. Install Ricoh UVC driver 1.0.1 or higher. Make sure your camera is in live streaming mode");
		}

		Renderer rend = this.GetComponentInChildren<Renderer>();
		WebCamTexture mycam = new WebCamTexture();

		mycam.deviceName = camName;
		rend.material.mainTexture = mycam;

		mycam.Play();

		// audio
		// this section working with HTC Vive, but have not 
		// verified spatial audio. Maybe try STEAM AUDIO?
		// https://valvesoftware.github.io/steam-audio/downloads.html
		// It's good enough for telepresence demo right now, but
		// I would like to tune the spatial audio
		//

		audioSource = GetComponent<AudioSource>();
		string[] audioDevices = Microphone.devices;

		for (int i = 0; i < audioDevices.Length; i++)
		{
			Debug.Log(i + " " + audioDevices[i]);
		}
		audioSource.clip = Microphone.Start(audioDevices[THETA_V_AUDIO_NUMBER], true, 10, 44100);
		audioSource.loop = true;
		while (!(Microphone.GetPosition(null) > 0 )) { }
		audioSource.Play();
	}
}

Using Default Audio Settings

image


#52

I think I might have forgot the last curly bracket. I just posted a new script with the audio working.

This is what I have when outputting to HTC Vive

image

My recording devices
image

My script debug log from Unity

image

Note that the script is finding the THETA V first. Thus, in the script, it is array element zero and the HTC Vive microphone is array element one.


#53

I have an Oculus Rift
so 2 Mics see screenshot.

Problem is. Performance starts lagging (image) and there is still no audio :confused:
See Screenshot!) I did everything the way you did.
I also used “RICOH THETA V FullHD” in the script. still lagging…
Maybe you know what is missing… :slight_smile: thx a lot!