Troubleshooting v4l2loopback Issues on Jetson with Ubuntu 20.04 in ARM Environment

"Hi there,

I’m currently running Ubuntu 20.04 on a Jetson device in an ARM environment. I’ve encountered a problem where v4l2loopback doesn’t seem to be functioning correctly, which is preventing ./gst_loopback from working. When I run ./gst_viewer, the video display appears on my computer. However, when I try to execute ./gst_loopback or initiate roscore followed by setting rosparam set cv_camera/device_id 0 and running rosrun cv_camera cv_camera_node, the video display doesn’t open.

I suspect this might be a pipeline issue, which is a common problem in such cases. To help diagnose and resolve this, I’m attaching a picture of the pipeline from my gst_viewer.c file. Could you please suggest how I might fix this? Or if you think the problem lies elsewhere, could you advise me on the commands I should try?

Help me…


Do you have /dev/video0 set in your source code? The default is /dev/video1

Can you open /dev/video0 with vlc, ffplay or gstreamer?

What camera model are you using? Is it a X?

I ask for your understanding. english is not my native language, so if you don’t understand my answer, ask again please.


yes. I have /dev/video0


no. I can’t open…

camera model is theta z1 and I don’t know if it’s necessary, jetson model is NVIDIA jetson AGX orin.

If you’re using opencv as your goal, can you use gstthetauvc instead of the loopback?

See the last post.

Do you have multiple versions of libuvc installed on your system? The Linux driver for the THETA installs a patched version of libuvc. Verify that your OS didn’t already come with a libuvc version already installed. It may conflict with the patched version of libuvc.

In the sourcecode, did you change this line to /dev/video0?

oh… I’m sorry. I don’t upload my pipeline code. that line is almost same.


and I change this part

   else
         		pipe_proc = " decodebin ! autovideosink sync=false";

but the same error occurs.
and even in ./gst_viewer, the camera launch screen does not appear.

and ptpcam also doesn’t run

and i don’t know how to check if the libuvc matches my system. is it right?


please tell me more.


Is your theta mounted as a filesystem? Try to manually unmount it. Interesting that ptpcam doesn’t work. I guess gohoto2 also doesn’t work?

Did you compile v42loopback from source? You have an error about tainted kernel.

see names of file with string libuvc using apt-cache

apt-cache search libuvc
libuvc-dev - cross-platform library for USB video devices - development files
libuvc-doc - cross-platform library for USB video devices - documentation
libuvc0 - cross-platform library for USB video devices

see if installed

dpkg -l libuvc-dev
dpkg-query: no packages found matching libuvc-dev
dpkg -l libuvc0
dpkg-query: no packages found matching libuvc0

see names of files with apt list

apt list | grep libuvc
libuvc-dev/jammy 0.0.6-1.1 amd64
libuvc-doc/jammy,jammy 0.0.6-1.1 all
libuvc0/jammy 0.0.6-1.1 amd64

see installed packages with apt list --installed

apt list  --installed | grep libuvc

unmount THETA as filesystem on NVIDIA Jetson

Environment

lesunhe (person with problem)

theta model: Z1
firmware: unknown
hardware platform: jetson model is NVIDIA jetson AGX orin (ARM)
OS: Ubuntu 20.04

craig tests (working on Jan 18, 2024)

theta model: Z1
firmware: 3.10.2
hardware platform: x86
OS: Ubuntu 22.04

Craig Summary

test result comment
USB API with gphoto2 success needed to unmount camera
v4l2loopback success compiled from source. /dev/video2 on my system
vlc display success make sure v4l2loopback is correctly loaded into kernel prior to test
OpenCV success works stable for an hour. seems like it can keep working much longer
gphoto2 --auto-detect
Model                          Port                                            
----------------------------------------------------------
Ricoh Theta Z1 (MTP)           usb:005,004     
gphoto2 --summary
Camera summary:                                                                
Manufacturer: Ricoh Company, Ltd.
Model: RICOH THETA Z1
  Version: 3.10.2
  Serial Number: 10010104
Vendor Extension ID: 0x6 (1.10)
Vendor Extension Description: 

Capture Formats: 
Display Formats: Association/Directory, JPEG, MP4, DNG, Firmware
Supported MTP Object Properties:
	Association/Directory/3001: dc01/StorageID dc02/ObjectFormat dc04/ObjectS

sudo modprobe v4l2loopback
lsmod |grep v4l2loopback
v4l2loopback           49152  0
videodev              364544  3 videobuf2_v4l2,v4l2loopback,uvcvideo
lsusb
...
Bus 005 Device 004: ID 05ca:036d Ricoh Co., Ltd RICOH THETA Z1
...
/gst_viewer -l
No : Product            : Serial    
 0 : RICOH THETA Z1     : 10010104  

./gst_loopback 
start, hit any key to stop

image


testing with OpenCV

python ./canny_edge.py --video_device=2
Called with args:
Namespace(video_device=2, gsttheta='none')
OpenCV version: 4.9.0
Device Number: 2
attempting to use v4l2loopback on /dev/video 2

sample script


import sys
import argparse
from tokenize import String
import cv2
import numpy as np

def parse_cli_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--video_device", dest="video_device", 
                        help="Video device # of USB webcam (/dev/video?) [0]",
                        default=0, type=int)
    parser.add_argument("--gsttheta", dest="gsttheta", help="nvdec, auto, none", default="none")
    arguments = parser.parse_args()
    return arguments


# Open an external usb camera /dev/videoX
def open_theta_device(device_number):
    print(f"attempting to use v4l2loopback on /dev/video {device_number}")
    return cv2.VideoCapture(device_number)

# https://github.com/nickel110/gstthetauvc
# example uses hardware acceleration
def open_gst_thetauvc_nvdec():
    print("attempting hardware acceleration for NVIDIA GPU with gstthetauvc")
    return cv2.VideoCapture("thetauvcsrc \
    ! queue \
    ! h264parse \
    ! nvdec \
    ! gldownload \
    ! queue \
    ! videoconvert n-threads=0 \
    ! video/x-raw,format=BGR \
    ! queue \
    ! appsink")    

# without hardware acceleration
def open_gst_thetauvc_auto():
    return cv2.VideoCapture("thetauvcsrc \
        ! decodebin \
        ! autovideoconvert \
        ! video/x-raw,format=BGRx \
        ! queue ! videoconvert \
        ! video/x-raw,format=BGR ! queue ! appsink")

def read_cam(video_capture):
    if video_capture.isOpened():
        windowName = "main_canny"
        cv2.namedWindow(windowName, cv2.WINDOW_NORMAL)
        cv2.resizeWindow(windowName,1280,720)
        cv2.moveWindow(windowName,0,0)
        cv2.setWindowTitle(windowName,"RICOH THETA OpenCV Python Demo")
        showWindow=3  # Show all stages
        showHelp = True
        font = cv2.FONT_HERSHEY_PLAIN
        helpText="'Esc' to Quit, '1' for Camera Feed, '2' for Canny Detection, '3' for All Stages. '4' to hide help"
        edgeThreshold=40
        showFullScreen = False
        while True:
            if cv2.getWindowProperty(windowName, 0) < 0: # Check to see if the user closed the window
                # This will fail if the user closed the window; Nasties get printed to the console
                break;
            ret_val, frame = video_capture.read();
            hsv=cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
            blur=cv2.GaussianBlur(hsv,(7,7),1.5)
            edges=cv2.Canny(blur,0,edgeThreshold)
            if showWindow == 3:  # Need to show the 4 stages
                # Composite the 2x2 window
                # Feed from the camera is RGB, the others gray
                # To composite, convert gray images to color. 
                # All images must be of the same type to display in a window
                frameRs=cv2.resize(frame, (640,360))
                hsvRs=cv2.resize(hsv,(640,360))
                vidBuf = np.concatenate((frameRs, cv2.cvtColor(hsvRs,cv2.COLOR_GRAY2BGR)), axis=1)
                blurRs=cv2.resize(blur,(640,360))
                edgesRs=cv2.resize(edges,(640,360))
                vidBuf1 = np.concatenate( (cv2.cvtColor(blurRs,cv2.COLOR_GRAY2BGR),cv2.cvtColor(edgesRs,cv2.COLOR_GRAY2BGR)), axis=1)
                vidBuf = np.concatenate( (vidBuf, vidBuf1), axis=0)

            if showWindow==1: # Show Camera Frame
                displayBuf = frame 
            elif showWindow == 2: # Show Canny Edge Detection
                displayBuf = edges
            elif showWindow == 3: # Show All Stages
                displayBuf = vidBuf

            if showHelp == True:
                cv2.putText(displayBuf, helpText, (11,20), font, 1.0, (32,32,32), 4, cv2.LINE_AA)
                cv2.putText(displayBuf, helpText, (10,20), font, 1.0, (240,240,240), 1, cv2.LINE_AA)
            cv2.imshow(windowName,displayBuf)
            key=cv2.waitKey(10)
            if key == 27: # Check for ESC key
                cv2.destroyAllWindows()
                break ;
            elif key==49: # 1 key, show frame
                cv2.setWindowTitle(windowName,"Camera Feed")
                showWindow=1
            elif key==50: # 2 key, show Canny
                cv2.setWindowTitle(windowName,"Canny Edge Detection")
                showWindow=2
            elif key==51: # 3 key, show Stages
                cv2.setWindowTitle(windowName,"Camera, Gray scale, Gaussian Blur, Canny Edge Detection")
                showWindow=3
            elif key==52: # 4 key, toggle help
                showHelp = not showHelp
            elif key==44: # , lower canny edge threshold
                edgeThreshold=max(0,edgeThreshold-1)
                print ('Canny Edge Threshold Maximum: ',edgeThreshold)
            elif key==46: # , raise canny edge threshold
                edgeThreshold=edgeThreshold+1
                print ('Canny Edge Threshold Maximum: ', edgeThreshold)
            elif key==74: # Toggle fullscreen; This is the F3 key on this particular keyboard
                # Toggle full screen mode
                if showFullScreen == False : 
                    cv2.setWindowProperty(windowName, cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_FULLSCREEN)
                else:
                    cv2.setWindowProperty(windowName, cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_NORMAL) 
                showFullScreen = not showFullScreen

    else:
     print ("camera open failed")



if __name__ == '__main__':
    arguments = parse_cli_args()
    print("Called with args:")
    print(arguments)
    print("OpenCV version: {}".format(cv2.__version__))
    print("Device Number:",arguments.video_device)
    if arguments.gsttheta=='nvdec':
        video_capture=open_gst_thetauvc_nvdec()
    elif arguments.gsttheta == 'auto':
        video_capture=open_gst_thetauvc_auto()
    else:
      video_capture=open_theta_device(arguments.video_device)
    read_cam(video_capture)
    video_capture.release()
    cv2.destroyAllWindows()

After “apt-cache search libuvc”,“dpkg -l libuvc-dev”, " dpkg -l libuvc0", “apt list | grep libuvc”, "apt list --installed | grep libuvc"command , the result is the same as yours.

and is it correct to check my firmware using the dmidecode command? maybe… 4.1 version.

I try to unmount Theta as filesystem following the manual you sent me and theta z1 camera recognized in gphoto2.



and ./gst_loopback can run!! thanks. but…

I see that example python file you sent doesn’t work. so I still don’t solve not operating camera z1 in opencv associated with v4l2…



If you plug a normal USB webcam into the Jetson, does it work with OpenCV?

Is there a reason that the pipeline for gst_loopback is different from the pipeline after the else statement? Did the nvv4l2decoder ! nv3dsink not work in the gst_loopback section?

Did you see this article about using gstthetauvc instead on the Jetson platform?

I tested gstthetauvc on Jetson Nano with OpenCV.

Hello, I’m sorry to be late in replying because I have time off from work on the weekends. Thank you for continuing for reply.

I plug a normal USB webcam into the Jetson, so it work with OpenCV.

I tried changing the pipeline, thennvv4l2decoder ! nv3dsink not working…
if this code run, theta device not found.

I tried using gstthetauvc but it also makes same error.

oh… my mistake.
gst-launch-1.0 thetauvcsrc mode=4K ! queue ! h264parse ! nvv4l2decoder ! queue ! nv3dsink sync=false working well.

so, How can I fix pipeline?

You could try and open an issue on GitHub for gstthetauvc. The developer definitely has a lot of expertise.

As you can stream to vlc, the problem is likely with some other software in your workflow.

For example, does your OpenCV have the gstreamer backend? Check it with the following:

image

You can also try searching on the NVIDIA formware for “ERROR: pipeline doesn’t want to preroll”

gst-launch-1.0 thetauvcsrc mode=4K ! queue ! h264parse ! nvv4l2decoder ! queue ! nv3dsink sync=false working well.

If that thetauvcsrc is working, can you explicitly specify nvv4l2decoder and nv3dsink in this pipeline?

VideoCapture cap("thetauvcsrc ! decodebin ! autovideoconvert ! video/x-raw,format=BGRx ! queue ! videoconvert ! video/x-raw,format=BGR ! queue ! appsink");

Does the pipeline above not work?