Linux 20.04.06 LTS: The THETA Z1 live streaming viewer works, but it does not function from loopback. I want to use it with OpenCV by recognizing it as a camera, but it is not recognized.
When connected via USB, the device is recognized normally and works fine with the viewer. However, when I run the loopback, it starts working but then keeps shutting down.
Could you please advise on how to proceed?
Do you have a device at /dev/video0
Do you have v4l2loopback loaded in the kernel?
You can check it with lsmod
lsmod
Module Size Used by
uvcvideo 88565 0
bnep 16562 2
zram 26166 4
overlay 48691 0
spidev 13282 0
v4l2loopback 37383 0
If you don’t have it in the kernel, load it with
modprobe v4l2loopback
Thank you. The loopback has been completed successfully.
The code execution briefly showed something on the screen, and an issue was detected, but I will try a bit more. Thank you for your response.
After adding the QoS settings for the loopback, the Python code started working properly.
@craig Thank you!
However, it seems like there’s more latency than I expected. Do you have any advice on how to address this?
lowest latency is approx 250ms. average is approx 350ms.
What latency are you seeing?
The first image is from the Python OpenCV demo, and the second image shows the latency of gst_viewer.
List hardware specs that you’re running the system on. Include information on GPU.
Do you have hardware acceleration enabled?
My test on x86 hardware with discrete NVIDIA GPU showing 250ms latency.
Are you using an NVIDIA GPU with CUDA?
Note that my pipeline looks like this:
if (strcmp(cmd_name, "gst_loopback") == 0)
// original pipeline
// pipe_proc = "decodebin ! autovideoconvert ! "
// "video/x-raw,format=I420 ! identity drop-allocation=true !"
// "v4l2sink device=/dev/video2 qos=false sync=false";
//
//modified pipeline below
pipe_proc = "nvdec ! gldownload ! videoconvert n-thread=0 ! "
"video/x-raw,format=I420 ! identity drop-allocation=true !"
"v4l2sink device=/dev/video2 qos=false sync=false";
Be aware that you may need to install some things on your system to use the pipeline above. Article covers required components.
Is your OpenCV compiled with CUDA and are you using CUDA?
This post and the ones around it discuss OpenCV bottlenecks and possibly using CUDA.
note that I did not progress very much using cv.cuda.
My computer specifications are as follows:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
When I run gst-inspect-1.0 --version as part of the steps in the “Overview of Steps” from this guide, the output is as follows:
Therefore, I matched the gst-plugins-bad version to 1.16.3, included the CODEC SDK version 12.2 in the gst-plugins-bad, ran NVENCODE_CFLAGS, and then proceeded with make. However, I encountered the following error:
Oh, I was using CUDA version 11.8, so I downgraded the SDK to version 11.~ and re-ran everything, and it works correctly now. However, there are still occasional frame glitches, but I’ll continue to refine the code.
@craig The latency has also significantly decreased. Thank you for your help!
Great that worked for you! Pretty cool.
thank you for reporting on the solution as it help help other people.
There are different possibilities for the frame glitch
other
