Theta Z1 - cv_camera/image_raw very low publishing frequency...

1

Hello all - I have been following this guide.

I am running

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 39 bits physical, 48 bits virtual
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 142
Model name: Intel(R) Core™ i5-8259U CPU @ 2.30GHz

ROS - Noetic

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ ./gst_loopback
start, hit any key to stop

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ ls /dev/video*
/dev/video0

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ rosparam set cv_camera/device_id 0

---------Camera is in LIVE mode on a USB 3-----------------------

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ rosrun cv_camera cv_camera_node
[ WARN:0] global …/modules/videoio/src/cap_gstreamer.cpp (935) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
[ INFO] [1702030691.368687351]: using default calibration URL
[ INFO] [1702030691.369622966]: camera calibration URL: file:///home/slam/.ros/camera_info/camera.yaml
[ INFO] [1702030691.369702540]: Unable to open camera calibration file [/home/slam/.ros/camera_info/camera.yaml]
[ WARN] [1702030691.369749604]: Camera calibration file /home/slam/.ros/camera_info/camera.yaml not found.

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ rostopic list
/cv_camera/camera_info
/cv_camera/image_raw
/cv_camera/image_raw/compressed
/cv_camera/image_raw/compressed/parameter_descriptions
/cv_camera/image_raw/compressed/parameter_updates
/cv_camera/image_raw/compressedDepth
/cv_camera/image_raw/compressedDepth/parameter_descriptions
/cv_camera/image_raw/compressedDepth/parameter_updates
/cv_camera/image_raw/theora
/cv_camera/image_raw/theora/parameter_descriptions
/cv_camera/image_raw/theora/parameter_updates
/rosout
/rosout_agg

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ rostopic echo /cv_camera/image_raw
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
average rate: 0.009
min: 111.611s max: 111.611s std dev: 0.00000s window: 2
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages
no new messages

image_view window will not open

slam@NUC8i5BEK:~/ros/catkin_ws/src/theta_z1/libuvc-theta-sample/gst$ rosrun image_view image_view image:=/cv_camera/image_raw
[ INFO] [1702031269.267831947]: Initializing nodelet with 8 worker threads.
[ INFO] [1702031269.343768532]: Using transport “raw”

Any tips would be greatly appreciated

Regards Mattias

2

There is some information on streaming here:

It does not cover ROS.

Can you get the stream to display on your computer with ./gst_viewer

You may be able to use gstthetauvc instead of v4l2loopback

3

Hi Craig and thank you for your reply.

Trying on another computer - still Ubuntu 20.04

Using gstthetauvc

slam@HP-ZBook-17-G4:~/ros/theta_z1/src/theta_z1$ gst-launch-1.0 thetauvcsrc mode=2K ! queue ! h264parse ! decodebin ! queue ! autovideosink sync=false
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...

It seems to work - but as soon as I launch the script, the blue light goes out on the camera and the LIVE icon starts blinking. Is this what I should expect?

My ROS node is failing to publish any video frames.

4

Try to isolate where the problem is occurring. First, try and get the camera to stream and display the frames on Linux as a test. If it works on Linux, then you can focus on the ROS communication problem.

Does it work with something like this:

gst-launch-1.0 thetauvcsrc mode=4K \
  ! queue \
  ! h264parse \
  ! nvdec \
  ! queue \
  ! glimagesink sync=false

Using with OpenCV and hardware acceleration.

import cv2
# pipeline below worked
# cap = cv2.VideoCapture("thetauvcsrc \
#     ! decodebin \
#     ! autovideoconvert \
#     ! video/x-raw,format=BGRx \
#     ! queue ! videoconvert \
#     ! video/x-raw,format=BGR ! queue ! appsink")

# pipeline suggestion thanks to nickel110
# attempt to force hardware acceleration
# tested with NVIDIA 510.73 with old GTX 950 on Ubuntu 22.04
cap = cv2.VideoCapture("thetauvcsrc \
    ! queue \
    ! h264parse \
    ! nvdec \
    ! gldownload \
    ! queue \
    ! videoconvert n-threads=0 \
    ! video/x-raw,format=BGR \
    ! queue \
    ! appsink")

if not cap.isOpened():
    raise IOError('Cannot open RICOH THETA')

while True:
    ret, frame = cap.read()
    frame = cv2.resize(frame, None, fx=0.25, fy=0.25, interpolation=cv2.INTER_AREA)
    cv2.imshow('frame', frame)


    c = cv2.waitKey(1)
    if c == 27:
        break

cap.release()
cv2.destroyAllWindows()