You can connect your Linux computer to a RICOH THETA with a USB cable and live stream 360 video at 4K, 30fps. Latency is between 250ms and 500ms with hardware decoding. Community tests verify that live streaming on Linux works with the newest RICOH THETA firmware versions available as of Sept 2021.
- RICOH THETA V firmware 3.70.1
- RICOH THETA Z1 firmware 2.00.1
To access the stream on your Linux computer, there is a patched version of libuvc.
A sample implementation is available:
There is also a plug-in for gstreamer
Extensive community documentation with examples is available:
There are 11 video tutorials on topics such as compiling the required software.
Active community discussion. It’s long, but there is a lot of great information if you keep scrolling.
The newest test with RICOH THETA V firmware 3.70.1 is shown with a Jetson Nano in the video below.
Test was done with JetPack 4.4.
I intend to redo the test with JetPack 4.6.
DetectNet works.
The training images are not fisheye, so you will need to retrain the model with fisheye images to get a reasonable test.
Information on fisheye image databases for use with AI training models that people in the community were working on in the past.
You may be able to use algorithms to modify an existing image database for use with the RICOH THETA. If you get success, please post.
Another approach is to extract sections from the 360 phone and compensate for distortion.
Live Streaming over USB on Ubuntu and Linux, NVIDIA Jetson - #89 by Jaap
Phone and work by Jaap de Vries
There’s some additional information on using cv.cuda and GpuMat in this post, but no successful conclusion on processing the frames in real-time.
Using Python with OpenCV works well.
If you’re using a Jetson Nano, you may need to reduce the resolution from 4K to 2K.
Testing v4l2loopback
There is a sample implementation to pipe the output to v4l2loopback and get the stream to appear on /dev/video*.
gst/gst_viewer
Decode and display sample using gstreamer. You may need gstreamer1.0 develpment packages to build and run.
To display RICOH THETA cameras connected to device.
$ ./gst_viewer -l
No : Product : Serial
0 : RICOH THETA Z1 : 10010104
load v4l2loopback
# modprobe v4l2loopback
verify v4l2loopback is loaded
$ lsmod
Module Size Used by
uvcvideo 88565 0
bnep 16562 2
zram 26166 4
overlay 48691 0
spidev 13282 0
v4l2loopback 37383 0
check for dummy video device.
$ v4l2-ctl --info
Driver Info (not using libv4l2):
Driver name : v4l2 loopback
Card type : Dummy video device (0x0000)
Bus info : platform:v4l2loopback-000
Driver version: 4.9.140
Capabilities : 0x85208003
start gst_loopback
$ ./gst_loopback
start, hit any key to stop
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Check video resolution.
$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'YU12'
Name : Planar YUV 4:2:0
Size: Discrete 1920x960
Interval: Discrete 0.033s (30.000 fps)
Testing with gst-launch-1.0 if there are two video cameras and the THETA is the second camera.
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw, framerate=30/1 ! xvimagesink
Power
The Z1 and V will slowly drain as you are streaming. For the Z1, you can use a BC 1.2 compliant hub attached to the Jetson Nano, RPi, or other Linux computer.
Community member nickel110 found other workarounds for low current problem on Linux USB3 port on some computers. However, I couldn’t get these to work in my tests on the Jetson Nano.
- If uvc device driver(uvcvideo) is loaded as a kernel module and you don’t use uvc device other than THETA,
disable uvc driver by adding uvc device driver to the driver blacklist.
e.g.
sudo sh -c "echo blacklist uvcvideo >> /etc/modprobe.d/blacklist.conf"
or
Before connecting the THETA, disable autosuspend feature of the USB driver by writing -1 to the
/sys/module/usbcore/parameters/autosuspend.
e.g.
sudo sh -c "echo -1 > /sys/module/usbcore/parameters/autosuspend"