Live Streaming over USB on Ubuntu and Linux, NVIDIA Jetson

I’m not sure it works with virtualbox. Do you have a spare NVIDIA Jetson lying around? Or, try booting the Windows computer from an external SSD directly into Linux.

It may work with virtualbox, but we have no reported success.

Do you have Virtualbox working with a normal webcam and something like OpenCV? If so, what device number is the existing webcam on the Linux OS?

Hi,
I tried in another computer with ubuntu and everything worked fine and streamed . but now in my main computer i installed ubuntu 20.04 and did exactly same and it says Theta not found. can you help on this issue. ?

How many webcams do you have listed on your main computer?

Post the contents of lsusb and ls -l /dev/video*

saad@sam:~$ v4l2-ctl --list-devices
Integrated Camera: Integrated C (usb-0000:05:00.3-3):
/dev/video0
/dev/video1

saad@sam:~$ lsusb
Bus 004 Device 003: ID 05ac:100e Apple, Inc. USB3.0 Hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 0bda:4852 Realtek Semiconductor Corp. Bluetooth Radio
Bus 003 Device 006: ID 05ca:2715 Ricoh Co., Ltd RICOH THETA Z1
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 002: ID 05e3:0620 Genesys Logic, Inc. USB3.2 Hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 048d:c101 Integrated Technology Express, Inc. ITE Device(8910)
Bus 001 Device 005: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 004: ID 045e:070f Microsoft Corp. LifeChat LX-3000 Headset
Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Hi @craig now it works on my work laptop as well. I dont know how but the stream has started. I thankyou for your immense support. I will be posting alot on this form :smiley:

1 Like

Good news.

To use v4l2loopback, you must run sudo modprobe v4l2loopback each time. Alternately, you can have it load automatically in /etc/modules-load.d/modules.conf.

You can verify that the v4l2loopback module is loaded with lsmod

$ lsmod
Module                  Size  Used by
bnep                   16562  2
zram                   26166  4
overlay                48691  0
spidev                 13282  0
v4l2loopback           37383  0

On one of your computers, you have two webcams, possibly there is an integrated webcam on the laptop.

In that case, the THETA will be at /dev/video1

Sometimes, the pipeline needs to be adjusted.

If the video freezes, try set qos=false

if (strcmp(cmd_name, "gst_loopback") == 0)
    pipe_proc = "decodebin ! autovideoconvert ! "
        "video/x-raw,format=I420 ! identity drop-allocation=true !"
        "v4l2sink device=/dev/video1 qos=false sync=false";

On Xavier, you may need to specify the decoder, instead of decodebin ! autovideoconvert The pipeline below is only for Jetson, not x86.

"nvv4l2decoder ! nv3dsink sync=false"

Update May 9, 2022 Running on Ubuntu 22.04

THETA X now running on Jammy Jellyfish, Ubuntu 22.04

nvidia driver 510.60

nvidia-smi
Mon May 9 15:24:06 2022
±----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02 Driver Version: 510.60.02 CUDA Version: 11.6

running wayland

echo $XDG_SESSION_TYPE
wayland


Switch to X session for v4l2loopback

RICOH THETA X running firmware 1.10.1

image

@craig
We faced a problem when we used JetsonNX+ThetaZ1. The gst_viewer works well, but the gst_loopback doesn’t work with an error msg. of internal data stream error.
Although we confirmed others also faced the same problem and solved that, we are still confusing how to fix this problem. Would you please give us some opinions?

Please confirm that you’re running v4l2loopback and loaded the module with sudo modprobe v4l2loopback

What is the output of lsmod |grep v4l2

Thank you very much for your reply.
We took the screenshot. We loaded the module and output also showed correctly lsmod |grep v4l2.
But the error still occurred.
Does any other considerable problem exist?

What is the output of lsusb? Does it show 2715 for the Z1?
image

In gst_viewer.c, does your /dev/video0 match the single webcam on your Jetson?

craig@craig-desktop:~$ ls -l /dev/video*
crw-rw----+ 1 root video 81, 0 May 13 06:23 /dev/video0

craig@craig-desktop:~/Development/ricoh/libuvc-theta-sample/gst$ date
Fri May 13 06:26:37 AM PDT 2022
craig@craig-desktop:~/Development/ricoh/libuvc-theta-sample/gst$ ./gst_loopback 
start, hit any key to stop

Thank you for your reply.
We checked what you mentioned.
First, we executed lsusb, the output was 2715. But the device name only showed Ricoh Co., Ltd without the camera type.

Second, in the gst_viewer.c was correctly edited, /dev/video0 was shown.

What version of JetPack are you running?

In the pipeline in the gst_viewer.c code, try to replace

  • decodebin with nvv4l2decoder
  • autovideoconvert with nvvidconv

I have my Jetson in the office and I’m at my home office today. I can try and test it again tomorrow.

Thank you for your reply.
Our JetPack Version is 4.5.1.


We tried to instead those codes as you mentioned in gst_viewer.c and then ran the command again. It shows that gst_loopback opening in blocking mode and segmentation fault.

Does it work with this:

	pipe_proc = "nvv4l2decoder ! autovideoconvert !"

Unfortunately, I don’t have an NX, but the nvv4l2decoder ! autovideoconvert ! works on the Nano.

What firmware version are you running in the Z1? Are you running version 2.0 in the Z1?

image

In this example, the theta command is the same as ptpcam in libgphoto2

image

Have you tried streaming on an x86 Linux machine for comparison?


Ref: https://developer.nvidia.com/embedded/jetpack-archive

lshw -C system
WARNING: you should run this program as super-user.
jetson                      
    description: Computer
    product: NVIDIA Jetson Nano Developer Kit
    serial: 
    width: 64 bits
    capabilities: smp cp15_barrier setend swp

image

Using VLC on /dev/video0

 cat /etc/nv_tegra_release 
# R32 (release), REVISION: 4.4, GCID: 23942405, BOARD: t210ref, EABI: aarch64, DATE: Fri Oct 16 19:44:43 UTC 2020

We tried your codes, but the problem still exits shown as below.


The segmentation fault still occurred. We confirmed the firmware version of 2.00.1.

Now we haven’t got a 32-bit Linux PC, so we can’t compare with it instantly.

Are you running the Jetson with ROS?

If so, maybe try this?

Change the product ID to the Z1, which is 2715. The example is for the V, which is 2712.

@iketani_takuya

Using this GitHub - nickel110/gstthetauvc: Gstreamer theta uvc plugin

I am using this pipeline successfully on x86 Ubuntu 22.04. I am using X11, not Wayland.

gst-launch-1.0 thetauvcsrc mode=4K ! queue ! h264parse ! nvdec ! queue ! glimagesink sync=false 

The plug-in is installed in /usr/lib/x86_64-linux-gnu/gstreamer-1.0

image

I am running NVIDIA driver 510.73.05

latency

about 360ms latency. Stream is stable.

@craig Its working perfectly!
But when I try to use the provided gst-launch parameters

thetauvcsrc mode=4K ! queue ! h264parse ! nvdec ! queue ! glimagesink sync=false 

in the OpenCV, it crashes.
Do you have some example for how to Read into OpenCV ?

1 Like

First, great to hear that you have gstthetauvc working. Congratulations. :slight_smile:

Second, I didn’t try to get this read into OpenCV yet.

nickel110 has this example:

VideoCapture cap("thetauvcsrc ! decodebin ! autovideoconvert ! video/x-raw,format=BGRx ! queue ! videoconvert ! video/x-raw,format=BGR ! queue ! appsink");

I have not tried this because it requires the gstreamer support to be built into OpenCV.

It’s possible that the OpenCV that comes with Ubuntu already has gstreamer support. However, I am not sure how to check on this.

For me, the first step would be to organize articles on building gstreamer support into OpenCV.

@craig Thank you for your answer!
nickel110 OpenCV example is working perfectly on my side, but my question is that how to make work the nvdec (Optimization - RICOH THETA Development on Linux)

gst-launch-1.0 thetauvcsrc mode=4K ! queue ! h264parse ! nvdec ! queue ! glimagesink sync=false 

with OpenCV.