Viewing 360° Video in Real-Time from THETA V with Oculus Go Browser
Original article in Japanese by @massie_g (Masashi Ganeko), Infocom Corp.
Translation by @jcasman. Many thanks to Masashi for posting this information!
Introduction
Oculus Go’s browser is based on Chromium and supports WebGL. WebVR can be used by using WebVR library A-FRAME.
Source: “Summary of Oculus Go browser from a Web Developer’s Perspective” [JAPANESE]
Also, real-time communications using WebRTC are possible based on experiments by @gtk2k and others.
TWEET: If you have Oculus Go, I’d like to check if the browser of Oculus Go supports WebRTC and getUserMedia, so please access the following address from Oculus Go’s browser. https://turbographics2000.github.io/webrtc_simplecheck/ If everything’s green, then it’s OK.
TWEET: It looks like this. Once you tap it, the microphone’s permission dialog will appear, and if you allow it, the loopback also comes through.
RETWEET: Oculus Go WebRTC support works!
TWEET: Success! I can voice chat between PC and Oculus Go.
Therefore, if you use a RICOH THETA V which can be recognized as a web camera via a USB connection, you should be able to see real time 360 degree video with the Oculus Go browser.
Try Moving Samples
I made samples in order to experience THETA V real-time 360 degree video with Oculus Go. It is published on GitHub Pages.
-
Main Samples Page: https://mganeko.github.io/aframe/ [JAPANESE and ENGLISH]
-
Page for broadcasting from PC: https://mganeko.github.io/aframe/pc.html
-
Page for viewing from Oculus Go: https://mganeko.github.io/aframe/go.html
-
-
Samples source code: https://github.com/mganeko/aframe
Broadcasting from a PC
-
Connect RICOH THETA V to the PC via USB, turn on the power and set it to Live mode
-
Open https://mganeko.github.io/aframe/pc.html with Chrome browser on your PC
-
The room is randomly allocated. This is possible to change.
-
By attaching ?apikey=ZZZZZZZZZ to the URL, you can use your SkyWay API KEY (Please allow it to use in the mganeko.github.io domain beforehand)
-
-
Click the [Get Devices] button to get a list of available Video devices (camera) and Audio devices (microphone)
-
Please be advised that you will be asked for access to the camera and microphone
-
Once THETA V is found, it will be selected automatically
-
If it is not detected even when the THETA V is connected, please exit Chrome and reboot again
-
-
Please click the “Start Video” button
- Video and audio are acquired and displayed in the browser
-
Please click the “Connect” button
-
Connect to SkyWay and join the designated Room
-
The URL for connecting with Oculus Go is displayed below your video
-
If you open this URL in Oculus Go browser (or another WebVR compatible browser) you can watch the video
-
- To stop broadcasting, click [Disconnect] → [Stop Video] in that order
Viewing through Oculus Go
-
Please start up your Oculus Go browser (or another WebVR compatible browser)
-
The URL displayed on the broadcast side is accessed by the browser
-
It’s recommended for saving time and effort to bookmark https://mganeko.github.io/aframe/go.html?room= beforehand and then you only have to add the room name at the end
-
By attaching ?apikey=ZZZZZZZZZ to the URL, you can use your SkyWay API KEY
-
Or you can open https://mganeko.github.io/aframe/g.html, enter the room name, and then click the Send button (the apikey can not be specified in this case)
-
-
It will take a little while to load
- When A-FRAME is ready, a message will be displayed in the center (Please click the window to start 360 video streaming)
-
Please click the screen (pull the trigger on the Oculus Go controller)
-
Connect to SkyWay and join the designated Room
-
The image from the THETA V 360 connected to the PC is displayed
-
-
Clicking the glasses type VR mode button on the bottom right changes it to 360 degree mode (align the cursor and pull the trigger on the Oculus Go controller)
- Depending on the direction your head is facing, the viewing direction of the image changes
-
To exit 360 degree mode, press the [Back] button on the controller
-
The process of disconnecting from the Skyway connection is not completed yet. Please reload or move to another page
Displaying 360° Videos using A-FRAME
A-FRAME is a framework that lowers the barriers to WebVR. Just even using Oculus Go’s browser, you can see A-FRAME samples in 360 degree 3D. Some of the samples even show saved 360° videos.
Also, even though it is not Oculus Go, there are attempts at displaying RICOH R’s videos on WebVR using A-FRAME.
- Try Theta R/RICOH R WebVR video in UVC mode [JAPANESE]
Based on this article, it was possible to see 360 degree real time videos in WebVR. It seems possible to do this.
On 360 Degree Video Formats
Roughly divided, there are two types of 360 degree images
-
Dual fisheye Format (2 fisheye images) - This is the format when connecting by USB to THETA S
-
Equirectangular Format (panorama) - This is the format when connecting by USB to THETA V, RICOH R
Since A-FRAME is compatible with Equirectangular Format, THETA S cannot be used as-is. Although it is unconfirmed, if RICOH THETA UVC Blender is used, it seems that output from THETA S could be converted into equirectangular format and used.
Display of 360 Degree Video using A-FRAME
Exact details will be left to another A-FRAME focused article, but in order to display 360 degree video, the following code can be used.
<a-scene>
<a-assets>
<video id="video1" autoplay loop crossorigin="anonymous" playsinline webkit-playsinline></video>
</a-assets>
<a-videosphere src="#video1" rotation="0 180 0">
</a-videosphere>
</a-scene>
Write the original video tag in the a-assets tag and reference the id of the video tag from the a-videosphere component. Normally we would specify the URL in video.src, but if we use WebRTC we will pass MediaStream to video.srcObject.
Version of A-FRAME
The latest version of A-FRAME is 0.8.0 (as of May 2018). However, as much as I tested with this version 360 degrees video could not be displayed well. I tried 0.7.1 and because I can display without a problem, I am using that.
About Using Something other than Oculus Go to View
A-FRAME can be used with other devices as long as it has a browser that supports WebVR.
-
Chrome or Firefox on PC (Windows / Mac / Linux)
- Although it is unconfirmed, you should be able to see 360 degree VR on Oculus Rift or HTC Vive
-
Chrome/Firefox on Android
- You can see 360 degree VR using Google Cardboard
-
Chrome/Firefox/Safari on iOS
- Using Google Cardboard feels like it almost works. The address bar disappears. I found the solution, you can view 360 degree VR using the combination of iOS and Google Cardboard: “Ultra-handy VR development framework A-FRAME Introduction: 1st Extra edition - iPhone address bar problem” [JAPANESE]
(Since I changed the model to iPhone 6, the characters were small and it was hard to see, so I used the magnified display mode, which was actually the cause. For those who suffer from the same problem, I will describe how to change the setting.)
No, it’s not close to the convenience of Oculus Go, but it can be done with other devices.
Video / Voice Communication Using WebRTC
When it comes to real-time video / voice communication with a web browser, WebRTC is the answer. Signaling servers and STUN / TURN servers are required. Although it can be setup by oneself, this time we used a service offered by NTT Communications, prioritizing actual speeds.
There are quite a few articles about using WebRTC. It’s a little bit old, but I wrote up some information on this, so please have a look.
- WebRTC Introduction 2016 (HTML5experts.jp) [JAPANESE]
Acquisition and Selection of Video / Audio devices
Although not mentioned in the WebRTC Introduction 2016 link up above, you can detect the video device (camera) and audio device (microphone) connected to the PC.
function getDevice() {
// Acquire video and audio to obtain the name of camera and microphone (ask user for permission)
// -- enum device --
navigator.mediaDevices.getUserMedia({video: true, audio: true})
.then(stream => callBackDeviceList(stream))
.catch(err => console.error(err));
}
// When you receive permission from the user, obtain a list of devices
function callBackDeviceList(stream) {
navigator.mediaDevices.enumerateDevices()
.then(function(devices) {
devices.forEach(function(device) {
if (device.kind === 'videoinput') {
const id = device.deviceId;
const label = device.label || 'camera' + '(' + id + ')';
// .... Processing when video device is detected
}
else if (device.kind === 'audioinput') {
const id = device.deviceId;
const label = device.label || 'microphone' + '(' + id + ')';
// .... Processing when audio device is detected
}
});
})
.catch(function(err) {
console.error("getDevices ERROR:", err);
});
stopLocalStream(stream);
}
// Stop video and audio
function stopLocalStream(stream) {
let tracks = stream.getTracks();
for (let track of tracks) {
track.stop();
}
}
To specify the Video / Audio device, use the detected id.
-
Audio device id: xxxxxxx
-
Video device id: yyyyyyy
- Desired width, height: wwww, hhhh
With this information, specify as follows:
let options = {
audio : {
deviceId : {exact: "xxxxxx" }
},
video : {
deviceId : {exact: "yyyyyy" },
width : { min : wwww, max : wwww },
height : { min : hhhh, max : hhhh }
}
}
navigator.mediaDevices.getUserMedia(options)
.then( stream => {
// ... Successful acquisition of media stream
})
.catch( err => {
// ... When there’s an error
});
Conclusion
Unity and other applications are required to develop Oculus Go applications, but WebVR can be developed using just a text editor. Real time communication is also possible with WebRTC. If you use A-FRAME, you can also use devices other than Oculus Go. Please try it out!
Extra Resources
If you’ve registered with the Partner Program (putting your THETA in developer mode) and have sent in your serial number (allowing you to develop and upload plug-ins), I’ve got some super useful resources for you!
theta360.guide has pulled together a Top 10 THETA Developer Tips, pulling together feedback from developers, saving you time. We also have some choice apks that you can load immediately and try out with documentation and source code.
Work with speaker volume, record audio files, build a Web GUI for your THETA and lots more!
Send us your contact info and you’ll get an email will all the download links.
