You can connect your Linux computer to a RICOH THETA with a USB cable and live stream 360 video at 4K, 30fps. Latency is between 250ms and 500ms with hardware decoding. Community tests verify that live streaming on Linux works with the newest RICOH THETA firmware versions available as of Sept 2021.
- RICOH THETA V firmware 3.70.1
- RICOH THETA Z1 firmware 2.00.1
To access the stream on your Linux computer, there is a patched version of libuvc.
A sample implementation is available:
There is also a plug-in for gstreamer
Extensive community documentation with examples is available:
There are 11 video tutorials on topics such as compiling the required software.
Active community discussion. It’s long, but there is a lot of great information if you keep scrolling.
The newest test with RICOH THETA V firmware 3.70.1 is shown with a Jetson Nano in the video below.
Test was done with JetPack 4.4.
I intend to redo the test with JetPack 4.6.
The training images are not fisheye, so you will need to retrain the model with fisheye images to get a reasonable test.
Information on fisheye image databases for use with AI training models that people in the community were working on in the past.
You may be able to use algorithms to modify an existing image database for use with the RICOH THETA. If you get success, please post.
Another approach is to extract sections from the 360 phone and compensate for distortion.
Phone and work by Jaap de Vries
There’s some additional information on using cv.cuda and GpuMat in this post, but no successful conclusion on processing the frames in real-time.
Using Python with OpenCV works well.
If you’re using a Jetson Nano, you may need to reduce the resolution from 4K to 2K.
There is a sample implementation to pipe the output to v4l2loopback and get the stream to appear on
Decode and display sample using gstreamer. You may need gstreamer1.0 develpment packages to build and run.
To display RICOH THETA cameras connected to device.
$ ./gst_viewer -l No : Product : Serial 0 : RICOH THETA Z1 : 10010104
# modprobe v4l2loopback
$ lsmod Module Size Used by uvcvideo 88565 0 bnep 16562 2 zram 26166 4 overlay 48691 0 spidev 13282 0 v4l2loopback 37383 0
check for dummy video device.
$ v4l2-ctl --info Driver Info (not using libv4l2): Driver name : v4l2 loopback Card type : Dummy video device (0x0000) Bus info : platform:v4l2loopback-000 Driver version: 4.9.140 Capabilities : 0x85208003
$ ./gst_loopback start, hit any key to stop Opening in BLOCKING MODE NvMMLiteOpen : Block : BlockType = 261 NVMEDIA: Reading vendor.tegra.display-size : status: 6 NvMMLiteBlockCreate : Block : BlockType = 261
Check video resolution.
$ v4l2-ctl --list-formats-ext ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'YU12' Name : Planar YUV 4:2:0 Size: Discrete 1920x960 Interval: Discrete 0.033s (30.000 fps)
gst-launch-1.0 if there are two video cameras and the THETA is the second camera.
gst-launch-1.0 v4l2src device=/dev/video1 ! video/x-raw, framerate=30/1 ! xvimagesink
The Z1 and V will slowly drain as you are streaming. For the Z1, you can use a BC 1.2 compliant hub attached to the Jetson Nano, RPi, or other Linux computer.
Community member nickel110 found other workarounds for low current problem on Linux USB3 port on some computers. However, I couldn’t get these to work in my tests on the Jetson Nano.
- If uvc device driver(uvcvideo) is loaded as a kernel module and you don’t use uvc device other than THETA,
disable uvc driver by adding uvc device driver to the driver blacklist.
sudo sh -c "echo blacklist uvcvideo >> /etc/modprobe.d/blacklist.conf"
Before connecting the THETA, disable autosuspend feature of the USB driver by writing -1 to the
sudo sh -c "echo -1 > /sys/module/usbcore/parameters/autosuspend"