I’m currently running Ubuntu 20.04 on a Jetson device in an ARM environment. I’ve encountered a problem where v4l2loopback doesn’t seem to be functioning correctly, which is preventing ./gst_loopback from working. When I run ./gst_viewer, the video display appears on my computer. However, when I try to execute ./gst_loopback or initiate roscore followed by setting rosparam set cv_camera/device_id 0 and running rosrun cv_camera cv_camera_node, the video display doesn’t open.
I suspect this might be a pipeline issue, which is a common problem in such cases. To help diagnose and resolve this, I’m attaching a picture of the pipeline from my gst_viewer.c file. Could you please suggest how I might fix this? Or if you think the problem lies elsewhere, could you advise me on the commands I should try?
If you’re using opencv as your goal, can you use gstthetauvc instead of the loopback?
See the last post.
Do you have multiple versions of libuvc installed on your system? The Linux driver for the THETA installs a patched version of libuvc. Verify that your OS didn’t already come with a libuvc version already installed. It may conflict with the patched version of libuvc.
In the sourcecode, did you change this line to /dev/video0?
Is your theta mounted as a filesystem? Try to manually unmount it. Interesting that ptpcam doesn’t work. I guess gohoto2 also doesn’t work?
Did you compile v42loopback from source? You have an error about tainted kernel.
see names of file with string libuvc using apt-cache
apt-cache search libuvc
libuvc-dev - cross-platform library for USB video devices - development files
libuvc-doc - cross-platform library for USB video devices - documentation
libuvc0 - cross-platform library for USB video devices
see if installed
dpkg -l libuvc-dev
dpkg-query: no packages found matching libuvc-dev
dpkg -l libuvc0
dpkg-query: no packages found matching libuvc0
see names of files with apt list
apt list | grep libuvc
libuvc-dev/jammy 0.0.6-1.1 amd64
libuvc-doc/jammy,jammy 0.0.6-1.1 all
libuvc0/jammy 0.0.6-1.1 amd64
see installed packages with apt list --installed
apt list --installed | grep libuvc
unmount THETA as filesystem on NVIDIA Jetson
Environment
lesunhe (person with problem)
theta model: Z1 firmware: unknown hardware platform: jetson model is NVIDIA jetson AGX orin (ARM) OS: Ubuntu 20.04
python ./canny_edge.py --video_device=2
Called with args:
Namespace(video_device=2, gsttheta='none')
OpenCV version: 4.9.0
Device Number: 2
attempting to use v4l2loopback on /dev/video 2
sample script
import sys
import argparse
from tokenize import String
import cv2
import numpy as np
def parse_cli_args():
parser = argparse.ArgumentParser()
parser.add_argument("--video_device", dest="video_device",
help="Video device # of USB webcam (/dev/video?) [0]",
default=0, type=int)
parser.add_argument("--gsttheta", dest="gsttheta", help="nvdec, auto, none", default="none")
arguments = parser.parse_args()
return arguments
# Open an external usb camera /dev/videoX
def open_theta_device(device_number):
print(f"attempting to use v4l2loopback on /dev/video {device_number}")
return cv2.VideoCapture(device_number)
# https://github.com/nickel110/gstthetauvc
# example uses hardware acceleration
def open_gst_thetauvc_nvdec():
print("attempting hardware acceleration for NVIDIA GPU with gstthetauvc")
return cv2.VideoCapture("thetauvcsrc \
! queue \
! h264parse \
! nvdec \
! gldownload \
! queue \
! videoconvert n-threads=0 \
! video/x-raw,format=BGR \
! queue \
! appsink")
# without hardware acceleration
def open_gst_thetauvc_auto():
return cv2.VideoCapture("thetauvcsrc \
! decodebin \
! autovideoconvert \
! video/x-raw,format=BGRx \
! queue ! videoconvert \
! video/x-raw,format=BGR ! queue ! appsink")
def read_cam(video_capture):
if video_capture.isOpened():
windowName = "main_canny"
cv2.namedWindow(windowName, cv2.WINDOW_NORMAL)
cv2.resizeWindow(windowName,1280,720)
cv2.moveWindow(windowName,0,0)
cv2.setWindowTitle(windowName,"RICOH THETA OpenCV Python Demo")
showWindow=3 # Show all stages
showHelp = True
font = cv2.FONT_HERSHEY_PLAIN
helpText="'Esc' to Quit, '1' for Camera Feed, '2' for Canny Detection, '3' for All Stages. '4' to hide help"
edgeThreshold=40
showFullScreen = False
while True:
if cv2.getWindowProperty(windowName, 0) < 0: # Check to see if the user closed the window
# This will fail if the user closed the window; Nasties get printed to the console
break;
ret_val, frame = video_capture.read();
hsv=cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
blur=cv2.GaussianBlur(hsv,(7,7),1.5)
edges=cv2.Canny(blur,0,edgeThreshold)
if showWindow == 3: # Need to show the 4 stages
# Composite the 2x2 window
# Feed from the camera is RGB, the others gray
# To composite, convert gray images to color.
# All images must be of the same type to display in a window
frameRs=cv2.resize(frame, (640,360))
hsvRs=cv2.resize(hsv,(640,360))
vidBuf = np.concatenate((frameRs, cv2.cvtColor(hsvRs,cv2.COLOR_GRAY2BGR)), axis=1)
blurRs=cv2.resize(blur,(640,360))
edgesRs=cv2.resize(edges,(640,360))
vidBuf1 = np.concatenate( (cv2.cvtColor(blurRs,cv2.COLOR_GRAY2BGR),cv2.cvtColor(edgesRs,cv2.COLOR_GRAY2BGR)), axis=1)
vidBuf = np.concatenate( (vidBuf, vidBuf1), axis=0)
if showWindow==1: # Show Camera Frame
displayBuf = frame
elif showWindow == 2: # Show Canny Edge Detection
displayBuf = edges
elif showWindow == 3: # Show All Stages
displayBuf = vidBuf
if showHelp == True:
cv2.putText(displayBuf, helpText, (11,20), font, 1.0, (32,32,32), 4, cv2.LINE_AA)
cv2.putText(displayBuf, helpText, (10,20), font, 1.0, (240,240,240), 1, cv2.LINE_AA)
cv2.imshow(windowName,displayBuf)
key=cv2.waitKey(10)
if key == 27: # Check for ESC key
cv2.destroyAllWindows()
break ;
elif key==49: # 1 key, show frame
cv2.setWindowTitle(windowName,"Camera Feed")
showWindow=1
elif key==50: # 2 key, show Canny
cv2.setWindowTitle(windowName,"Canny Edge Detection")
showWindow=2
elif key==51: # 3 key, show Stages
cv2.setWindowTitle(windowName,"Camera, Gray scale, Gaussian Blur, Canny Edge Detection")
showWindow=3
elif key==52: # 4 key, toggle help
showHelp = not showHelp
elif key==44: # , lower canny edge threshold
edgeThreshold=max(0,edgeThreshold-1)
print ('Canny Edge Threshold Maximum: ',edgeThreshold)
elif key==46: # , raise canny edge threshold
edgeThreshold=edgeThreshold+1
print ('Canny Edge Threshold Maximum: ', edgeThreshold)
elif key==74: # Toggle fullscreen; This is the F3 key on this particular keyboard
# Toggle full screen mode
if showFullScreen == False :
cv2.setWindowProperty(windowName, cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_FULLSCREEN)
else:
cv2.setWindowProperty(windowName, cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_NORMAL)
showFullScreen = not showFullScreen
else:
print ("camera open failed")
if __name__ == '__main__':
arguments = parse_cli_args()
print("Called with args:")
print(arguments)
print("OpenCV version: {}".format(cv2.__version__))
print("Device Number:",arguments.video_device)
if arguments.gsttheta=='nvdec':
video_capture=open_gst_thetauvc_nvdec()
elif arguments.gsttheta == 'auto':
video_capture=open_gst_thetauvc_auto()
else:
video_capture=open_theta_device(arguments.video_device)
read_cam(video_capture)
video_capture.release()
cv2.destroyAllWindows()
After “apt-cache search libuvc”,“dpkg -l libuvc-dev”, " dpkg -l libuvc0", “apt list | grep libuvc”, "apt list --installed | grep libuvc"command , the result is the same as yours.
Is there a reason that the pipeline for gst_loopback is different from the pipeline after the else statement? Did the nvv4l2decoder ! nv3dsink not work in the gst_loopback section?
Did you see this article about using gstthetauvc instead on the Jetson platform?
