I will try it tonight. One more question, I am running Ricoh on private wifi. When I start streaming all my programs connected to that network start responding very slow. The current resolution is 2k, I wanted to try different resolutions what are the parameters?
4K -THETAUVC_MODE_FHD_2997
2k - THETAUVC_MODE_UHD_2997
Help - RICOH THETA Development on Linux
September 1, 2020 - First Linux Meetup - RICOH THETA Development on Linux
minor modification to enable changing the resolution on the fly for testing.
Craig, the fix worked in x86.I will keep on working on it and keep you updated
Thanks , one thing I observed
./gst_loopback --format 4K or 2k always defaults to 4k
Had to change the code THETAUVC_MODE_UHD_2997 as you had suggested before
Hello everyone,
I hope you donāt mind me chiming in.
Iāve been trying to implement YOLO in realtime webcam with a Theta Z1 on a computer without access to CUDA.
For now I got it to compile and running but the frame rate is absolutely dead. Something like 0.07 fpsā¦ 
I think Iām not using my GPU because this also happens when Iām analyzing previously downloaded files so there is also room for improvement there. However, and the reason why Iām here, is because Iāve also noticed that although gst_viewer is very fast, when I use v4l2loopback and OpenCV it slows it down immensely.
So I find this discussion here about trying to use a plugin to use gstreamer directly and I tried it.
This is what I did:
Factory Details:
Rank none (0)
Long-name Theta UVC video source
Klass Generic
Description Reads live video from THETA S/V/Z1
Author Koji Takeo nickel110@icloud.com
Plugin Details:
Name thetauvc
Description Theta UVC plugin
Filename /usr/lib/x86_64-linux-gnu/gstreamer-1.0/gstthetauvc.so
Version 0.0.1
License LGPL
Source module gstthetauvc
Binary package GStreamer thetauvc plugin
Origin URL http://github.com/nickel110/gstthetauvc
GObject
Ā±āGInitiallyUnowned
Ā±āGstObject
Ā±āGstElement
Ā±āGstBaseSrc
Ā±āGstPushSrc
Ā±āGstThetauvcsrc
Pad Templates:
SRC template: āsrcā
Availability: Always
Capabilities:
video/x-h264
width: 3840
height: 1920
framerate: 30000/1001
stream-format: byte-stream
alignment: nal
profile: constrained-baseline
video/x-h264
width: 1920
height: 960
framerate: 30000/1001
stream-format: byte-stream
alignment: nal
profile: constrained-baseline
video/x-h264
width: 1920
height: 1080
framerate: 30000/1001
stream-format: byte-stream
alignment: nal
profile: high
Element has no clocking capabilities.
[ WARN:0] global /home/husarion/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (2056) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global /home/husarion/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (1034) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/husarion/opencv_build/opencv/modules/videoio/src/cap_gstreamer.cpp (597) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ WARN:0] global /home/husarion/opencv_build/opencv/modules/videoio/src/cap_v4l.cpp (890) open VIDEOIO(V4L2:/dev/video0): canāt open camera by index
Floating point exception (core dumped)
Do you know if I missed any step? Iām unsure where I should change VideoCapture pipeline as shown by Craig. So I think atleast that is missing
Best regards,
MƔrio
First, youāre very much welcome to post and it is appreciated, especially with all the great testing youāve done.
I have not actually used the gstreamer plug-in yet, but still intend to. I believe it works.
The developer that wrote it also wrote the driver. He updated gstthetauvc 10 days ago and he sent me email that heās using the gstreamer plug-in.
Support rotation only transformation. Ā· nickel110/gstthetauvc@c7db9fe Ā· GitHub
If I am too slow in testing the plug-in myself, you may want to try opening an issue. He may have free time. This is his personal project.
He also has a Canny demo that doesnāt need v4l2loopback
He sent me this advice on OpenCV
For OpenCV,
thetauvcsrc ! decodebin ! autovideoconvert ! video/x-raw,format=BGRx ! queue ! videoconvert ! video/x-raw,format=BGR ! queue ! appsinkwould work on most platforms.
My biggest problem right now is that I donāt know what to do next. Where do I put that thetauvcsrc ! decodebin ! autovideoconvert ! video/x-raw,format=BGRx ! queue ! videoconvert ! video/x-raw,format=BGR ! queue ! appsink
Iām a beginner to all this 
ā¦ My instinct was to check the python scripts for the detections themselves. For example I have one that detects faces and it has a part dedicated to capturing the video with the following using opencv:
#To capture video from webcam.
cap = cv2.VideoCapture(0)
Or is it somewhere in the openCv configuration?
Best,
MaM
I am having the same issue as the original post of this thread with my Jetston NX + Theta Z1
I have a Jetson NX fresh install Jetson NX Developer Kit SD Card Image w/ Jetpack 4.5.1 (ubuntu 18.04)
REF: i installed all the packages as per: RICOH THETA Development on Linux
$ sudo apt-get install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio libgstreamer-plugins-base1.0-dev
I installed three libuvc-theta GitHub repos below to compile the driver and sample code w/o any errors
when i connect Z1 to Jetson and power on (photo mode)
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ lsusb | grep -i ricoh
Bus 001 Device 007: ID 05ca:036d Ricoh Co., Ltd
when i put into live mode (see the word live under the camera icon)
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ lsusb | grep -i ricoh
Bus 001 Device 008: ID 05ca:2715 Ricoh Co., Ltd
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ cd ~/v4l2loopback
ubuntu@jetson-nx1:~/v4l2loopback$ lsmod | grep loopback
ubuntu@jetson-nx1:~/v4l2loopback$ sudo depmod -a
[sudo] password for ubuntu:
ubuntu@jetson-nx1:~/v4l2loopback$ sudo modprobe v4l2loopback
ubuntu@jetson-nx1:~/v4l2loopback$ lsmod | grep loopback
v4l2loopback 42831 0
ubuntu@jetson-nx1:~/v4l2loopback$ ls /dev/video*
/dev/video0
ubuntu@jetson-nx1:~/v4l2loopback$ v4l2-ctl --list-devices --list-formats
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video0
ioctl: VIDIOC_ENUM_FMT
ubuntu@jetson-nx1:~/v4l2loopback$ cd -
/home/ubuntu/libuvc-theta-sample/gst
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ ./gst_viewer -l
No : Product : Serial
0 : RICOH THETA Z1 : 14xxxxxxxxx
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ ./gst_viewer
Can't open THETA
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ gst-inspect-1.0 --gst-version
GStreamer Core Library version 1.14.5
Iām at a loss of what iām missing. Iāve gone through this thread trying to find what nugget i might have missed and not finding anything similar to my issue
In gst_viewer.c, confirm that you changed /dev/video1 to /dev/video0, assuming you only have one camera connected to the Jetson.
On Xavier, you may need this:
change ādecodebin ! autovideosink sync=falseā to ānvv4l2decoder ! nv3dsink sync=falseā
Thank you Craig,
Yes, i tried multiple options video0, video1, and even video7 (after doing modprobe v4l2loopback video_nr=7 just for grins). i did a āmake cleanā, āvi the .c fileā and rebuild with āmakeā . The fact that i dont see theta info in the printouts has me wondering if the driver is correctly seeing the camera.
ubuntu@jetson-nx1:~$ v4l2-ctl --list-devices --list-formats
Dummy video device (0x0000) (platform:v4l2loopback-000):
/dev/video0
ioctl: VIDIOC_ENUM_FMT
ubuntu@jetson-nx1:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
ubuntu@jetson-nx1:~$ v4l2-ctl -d 0 -D
Driver Info (not using libv4l2):
Driver name : v4l2 loopback
Card type : Dummy video device (0x0000)
Bus info : platform:v4l2loopback-000
Driver version: 4.9.201
Capabilities : 0x85208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x05208003
Video Capture
Video Output
Video Memory-to-Memory
Read/Write
Streaming
Extended Pix Format
ubuntu@jetson-nx1:~$
It should work on your platform.
Focus on gst_viewer first without v4l2loopback In the example above, do you have it set to /dev/video0 in the c code?
There are several other people on this thread that have it running on the Jetson Xavier NX with the same OS.
When you run gst_viewer, the camera ID needs to be 05ca:2715.
Can you run lsusb |grep -i ricoh immediately before running ./gst_viewer. detach all other USB cameras from the Jetson and just use /dev/video0 for now.
Thanks Craig,
I completely understand and have followed this on the thread. Here is what i see, yes, its 2715
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ make clean
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ cat gst_viewer.c | grep -A3 gst_loopback
if (strcmp(cmd_name, "gst_loopback") == 0)
pipe_proc = "decodebin ! autovideoconvert ! "
"video/x-raw,format=I420 ! identity drop-allocation=true !"
"v4l2sink device=/dev/video0 sync=false";
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ make
cc -pthread -I/usr/local/include -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/aarch64-linux-gnu/glib-2.0/include -I/usr/include/libusb-1.0 -c -o gst_viewer.o gst_viewer.c
cc -pthread -I/usr/local/include -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/aarch64-linux-gnu/glib-2.0/include -I/usr/include/libusb-1.0 -c -o thetauvc.o thetauvc.c
cc gst_viewer.o thetauvc.o -o gst_viewer -L/usr/local/lib -lgstapp-1.0 -lgstbase-1.0 -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -luvc -lusb-1.0 -pthread
ln -sf gst_viewer gst_loopback
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ ./gst_viewer -l
No : Product : Serial
0 : RICOH THETA Z1 : 14xxxxxxxxx
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ ./gst_viewer
Can't open THETA
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ lsusb
Bus 002 Device 002: ID 0bda:0489 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 13d3:3549 IMC Networks
Bus 001 Device 013: ID 05ca:2715 Ricoh Co., Ltd
Bus 001 Device 005: ID 04ca:0027 Lite-On Technology Corp.
Bus 001 Device 004: ID 0461:4d22 Primax Electronics, Ltd
Bus 001 Device 002: ID 0bda:5489 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$
Using the same cable, can you plug the Z1 into a x86 Windows machine or a Mac and confirm that the streaming works?
For Windows, you will need this driver.
You should be able to open it up on Windows/Mac with OBS or VLC.
On the Jetson NX, can you test normal 4K webcam (like a logitech that you have on your desktop)?
Possibilities to eliminate
| possible problem to test | test |
|---|---|
| USB cable is bad | try same cable on Windows or Mac. Swap cables |
| power source to Jetson is marginal | charge power source to something with more amps. try other 4K webcam on same USB port |
| Z1 streaming is bad | test on Windows or Mac |
the Z1 runs an OS inside of it. You can upgrade the camera firmware with the desktop app, called the āBasic appā
Yup, i went through that already. I have tested 3 different USB/USB-C cables
On Windows 10, the driver is RICOH THETA V/Z1 4K
https://youtu.be/qUzciWQ5HiM?t=190
However, if you have it working on the Mac, you donāt need to get it working on Windows. That was just to see whether the Z1 could stream to any device.
I donāt have a Xavier NX, so I canāt precisely replicate your test. At the playlist below, I tested it with Jetson Nano.
My original tests were on JetPack 4.4 (Ubuntu 18.04), Jetson Nano B01. I used a microSD card. I think it only makes a difference if you save the frames to local storage. To just run the camera, the microSD card seems fine.
Equipment - RICOH THETA Development on Linux
Thereās no open issues on the GitHub repo, so I feel we may be able to solve this one eventually. However, itās not obvious what the solution is.
I can try to install JetPack 4.5.1 on the Nano, but it may take me several days as I need to reconfigure my office. I moved the monitor, desk, all the gear I was using for the Nano into a closet when relatives came to visit. I was planning to take set up the Nano again anyway.
Just so we have a reference, is this is the device you are using with JetPack 4.5.1?
Maybe someone else on the forum can spot the problem. There are several people that are using the Jetson Xavier, though I think most of the tests were done with JetPack 4.4. That Xavier is a really cool device. So much power in a small package.
Thank you so much for the prompt replies.
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ cat /etc/nv_tegra_release
# R32 (release), REVISION: 5.1, GCID: 27362550, BOARD: t186ref, EABI: aarch64, DATE: Wed May 19 18:16:00 UTC 2021
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ cat /usr/local/cuda/version.txt
CUDA Version 10.2.89
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$ sudo apt-cache show nvidia-jetpack
[sudo] password for ubuntu:
Package: nvidia-jetpack
Version: 4.5.1-b17
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.5.1-b17), nvidia-opencv (= 4.5.1-b17), nvidia-cudnn8 (= 4.5.1-b17), nvidia-tensorrt (= 4.5.1-b17), nvidia-visionworks (= 4.5.1-b17), nvidia-container (= 4.5.1-b17), nvidia-vpi (= 4.5.1-b17), nvidia-l4t-jetson-multimedia-api (>> 32.5-0), nvidia-l4t-jetson-multimedia-api (<< 32.6-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.5.1-b17_arm64.deb
Size: 29372
SHA256: 378f7588e15c35692eb1bed6f336be74f4f396d88fad45af67c68e22b63be04b
SHA1: e41f26a3d8326e9952915eee12fa37e17de3245f
MD5sum: 31b2bd9d0f214f74acaeb3d8e4279e9d
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8
Package: nvidia-jetpack
Version: 4.5-b129
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.5-b129), nvidia-opencv (= 4.5-b129), nvidia-cudnn8 (= 4.5-b129), nvidia-tensorrt (= 4.5-b129), nvidia-visionworks (= 4.5-b129), nvidia-container (= 4.5-b129), nvidia-vpi (= 4.5-b129), nvidia-l4t-jetson-multimedia-api (>> 32.5-0), nvidia-l4t-jetson-multimedia-api (<< 32.6-0)
Homepage: http://developer.nvidia.com/jetson
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.5-b129_arm64.deb
Size: 29360
SHA256: 002646e6d81d13526ade23d7c45180014f3cd9e9f5fb0f8896b77dff85d6b9fe
SHA1: cb17547b902b2793e0df86d561809ecdbf7e401f
MD5sum: 06962c42e462f643455d6194d1a2d641
Description: NVIDIA Jetpack Meta Package
Description-md5: ad1462289bdbc54909ae109d1d32c0a8
ubuntu@jetson-nx1:~/libuvc-theta-sample/gst$
āÆ curl -d '{"name": "camera.reset"}' -H "Content-Type: application/json" -X POST http://192.168.1.1/osc/commands/execute
{"name":"camera.reset","state":"done"}%
not sure what to try nextā¦
On Windows, you need to use a software device driver. You need to install the Windows device from the web site listed. Download | RICOH THETA
I suggest you verify that the Z1 can stream to either a Mac or windows 10 machine. If it cannot stream, suggest you call the official RICOH customer support and consider a replacement, hopefully under warranty. Contact | RICOH THETA
Can you take a picture or short video (with a mobile phone or web cam) of the Jetson you are using plugged into the THETA camera in live mode? If itās a video, you may need to put the file on Google Drive or some other storage and put the link here.
The camera is widely used on quadruped robots for prototyping or small-volume robots.
Hi Craig,
Thanks so much for your infomation.
I have a jetson nano board and I want to push theta z1 stream from the board to a url,
so in gst_viewer.c, I changed ādecodebin ! device=/dev/video0 sync=falseā to ānv3dsink device=/dev/video0 sync=falseā and run ā./gst_loopbackā, it works fine!
but after I opened another command window to use ffmpeg command to push stream, error occurred. It shows /dev/video0: Device or resource busy.
Is there anything I should do before running ffmpeg command?
Or is there any other solution to push stream from theta z1 to some other url?
Thank you.
To use ffmpeg, from the command line, we are using v4l2loopback to get the stream on /dev/video*
Example from Paul Gullet
ffmpeg -f lavfi -i anullsrc \
-f v4l2 -s 3480x1920 -r 10 -i /dev/video0 \
-vcodec libx264 -pix_fmt yuv420p -preset ultrafast \
-strict experimental -r 25 -g 20 -b:v 2500k \
-codec:a libmp3lame -ar 44100 -b:a 11025 -bufsize 512k \
-f flv rtmp://a.rtmp.youtube.com/live2/secret-key
Example from my own test
ffmpeg -f lavfi -i anullsrc -f v4l2 -s 1920x960 -r 10 -i /dev/video2 \
-vcodec libx264 -pix_fmt yuv420p \
-b:v 2500k \
-codec:a libmp3lame -ar 44100 -b:a 11025 -bufsize 512k \
-f flv rtmp://a.rtmp.youtube.com/live2/$SECRET_KEY
Software - RICOH THETA Development on Linux
$ lsmod
Module Size Used by
bnep 16562 2
zram 26166 4
overlay 48691 0
spidev 13282 0
v4l2loopback 37383 0
nvgpu 1579891 18
bluedroid_pm 13912 0
ip_tables 19441 0
x_tables 28951 1 ip_tables
craig@jetson:/etc/modules-load.d$
What version of JetPack are you running on the Nano? Someone else is having a problem with getting the camera recognized on JetPack 4.5.1 on Jetson NX.
Verified streaming to YouTube with ffmpeg. Should work with other cloud-based services.
Hi Craig,
I have been trying to live stream from a Theta V on a Jetson Nano. I have followed your guide and I have made ./gst_viewer work. But When I try ./gst_loopvback, the terminal output āError: Internal data stream error.ā
I have installed v4l2loop and the kernel module has been loaded.
Do you know what happened?
JetPack Version: 4.6
Best regards
Zheng
