Live Streaming over USB on Ubuntu and Linux, NVIDIA Jetson

, ,
86

I put it into a loop to get the frames from RICOH THETA Z1 to see if it was using the GPU and I don’t think it is.

image

This screen grab is coming over an X forwarding session to another computer, so it may be faster on a monitor plugged into the Jetson. If I increase the window size, the latency gets worse. The interesting challenge is to get cv2.cuda.remap working, which I don’t have working.

equi_test

import cv2
import os
import Equirec2Perspec2 as E2P

print(f"OpenCV version {cv2.__version__}")
cap = cv2.VideoCapture(0)

# Check if the webcam is opened correctly
if not cap.isOpened():
    raise IOError("Cannot open webcam")

while True:
    ret, frame = cap.read()
    #print(frame.shape)    
    equ = E2P.Equirectangular(frame)
    frame = equ.GetPerspective(120, 180, -15, 400, 400)
    cv2.imshow('Input', frame)

    c = cv2.waitKey(1)
    if c == 27:
        break

cap.release()
cv2.destroyAllWindows()

in Equirec2Perspec2.py

class Equirectangular:
    def __init__(self, img):
        self._img = img
        [self._height, self._width, _] = img.shape

There’s a guy at the link below trying to use cv2.cuda.remap for the same thing we are.

In the comments, it ends with a happy “it’s working” with a plethora of exclamation points.

image
image719×194 37.1 KB

I’m not sure what this cv2.fisheye… is doing

map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), K, DIM, cv2.CV_16SC2)#

But, other than that, loading the map looks doable.

        #map1 = map1.astype(np.float32)
        #map2 = map2.astype(np.float32)
        #map1 = cv2.cuda_GpuMat(map1)
        #map2 = cv2.cuda_GpuMat(map2)

then at some point, he’s able to use cv2.cuda.resize with hopefully faster processing…

        #resized = cv2.cuda.resize(img2, (new_w, new_h), interpolation=cv2.INTER_LINEAR)

Another example of using cv.cuda.remap by the same guy, with another “it’s working” at the end

Some friendly guy posted this, which appears to work, according to the original poster.

cuDst = cv.cuda_GpuMat(cuMat.size(),cuMat.type())
cv.cuda.remap(cuMat,cuMapX,cuMapY,dst=cuDst,interpolation=cv.INTER_LINEAR)

We probably just need to read up on cv.cuda and the GpuMat and maybe we’ll get it too, like the happy guy posting on the OpenCV forums. :slight_smile:

2 Likes