OpenCv Ricoh Z1 stream, empty frame received

Hi all, I’m using a Ricoh Z1 with opencv on my ubuntu 20.04. I’ve followed the tutorial to get the streaming running and I can run the gst_loopback form the libuvc-theta-sample (link) succesfully. I get my equirectangular image from the ricoh in my /dev/video0. Then, when I want to get my video frame from openCV in c++ I receive an empty frame with (0,0) resolution. When I do the same with Python all works nice also, I can do it in c++ with a pre-recorded Z1 video and with a common webcam live, but not with the live stream of the Z1.

Has anyone encountered the same issue? A week ago I got it working with opencv in c++ but I don’t know what changed and now is impossible.

I attached the build information of my opencv :

Opencv version: 3.4.0
Buid Information
General configuration for OpenCV 3.4.1 =====================================
  Version control:               unknown

  Platform:
    Timestamp:                   2021-09-27T11:02:35Z
    Host:                        Linux 5.11.0-36-generic x86_64
    CMake:                       3.16.3
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (3 files):          + SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (1 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (4 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (8 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (0 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_SKX

  C/C++:
    Built as dynamic libs?:      YES
    C++11:                       YES
    C++ Compiler:                /usr/bin/c++  (ver 9.3.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      
    Linker flags (Debug):        
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          dl m pthread rt
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 calib3d core features2d flann highgui imgcodecs imgproc java java_bindings_generator objdetect shape stitching superres video videoio viz
    Disabled:                    dnn js ml photo python_bindings_generator ts world
    Disabled by dependency:      videostab
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev python2 python3
    Applications:                apps
    Documentation:               NO
    Non-free algorithms:         NO

  GUI: 
    GTK+:                        YES (ver 3.24.20)
      GThread :                  YES (ver 2.64.6)
      GtkGlExt:                  NO
    VTK support:                 YES (ver 7.1.1)

  Media I/O: 
    ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver )
    WEBP:                        /usr/lib/x86_64-linux-gnu/libwebp.so (ver encoder: 0x020e)
    PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.37)
    TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.1.0)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so (ver 2.3.0)

  Video I/O:
    DC1394:                      YES (ver 2.2.5)
    FFMPEG:                      YES
      avcodec:                   YES (ver 57.107.100)
      avformat:                  YES (ver 57.83.100)
      avutil:                    YES (ver 55.78.100)
      swscale:                   YES (ver 4.8.100)
      avresample:                NO
    GStreamer:                   
      base:                      YES (ver 1.16.2)
      video:                     YES (ver 1.16.2)
      app:                       YES (ver 1.16.2)
      riff:                      YES (ver 1.16.2)
      pbutils:                   YES (ver 1.16.2)
    libv4l/libv4l2:              NO
    v4l/v4l2:                    linux/videodev2.h
    gPhoto2:                     YES

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2017.0.3 [2017.0.3]
           at:                   /home/albert/www/dragonfly-installation-linux-2.3/dragonfly-linux/data/dependencies/opencv-3.4.1/build/3rdparty/ippicv/ippicv_lnx
    Intel IPP IW:                sources (2017.0.3)
              at:                /home/albert/www/dragonfly-installation-linux-2.3/dragonfly-linux/data/dependencies/opencv-3.4.1/build/3rdparty/ippicv/ippiw_lnx
    Lapack:                      NO
    Eigen:                       YES (ver 3.3.7)
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (no extra features)
    Include path:                /home/albert/www/dragonfly-installation-linux-2.3/dragonfly-linux/data/dependencies/opencv-3.4.1/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python (for build):            /usr/bin/python2.7

  Java:                          
    ant:                         /bin/ant (ver 1.10.7)
    JNI:                         /usr/lib/jvm/java-1.11.0-openjdk-amd64/include /usr/lib/jvm/java-1.11.0-openjdk-amd64/include/linux /usr/lib/jvm/java-1.11.0-openjdk-amd64/include
    Java wrappers:               YES
    Java tests:                  YES

  Matlab:                        NO

  Install to:                    /usr/local
-----------------------------------------------------------------

using this basic example to test it:

#include "opencv2/opencv.hpp"
#include <iostream>

using namespace std;
using namespace cv;

int main()
{
  // Create video object
  VideoCapture cap(0); 
  cv::Mat frame;
  cout<<"Is open: "<<cap.isOpened()<<endl;
  cout<<"Opencv version: "<<CV_VERSION<<endl;
  cout<<"Buid Information"<<getBuildInformation().c_str()<<endl;
  bool is_grabed = cap.grab();
  cout<<"is grabed: "<<is_grabed<<endl;
  while(true){
    cap.read(frame);  
    cout<<"Frame empty: "<<frame.empty()<<endl;
    imshow("frame", frame);
    waitKey(1);
  }
}

Many thanks!

I have not seen this problem before.

Consider trying to use gstthetauvc instead of libuvc-theta-sample and v4l2loopback

This may be unrelated, but isn’t OpenCV on version 4.5? Is there a reason you’re using version 3.4? Likely, it’s unrelated, but I noticed that on my own system, 22.04 (Jammy Jellyfish), I am using OpenCV 4.5.

Hi and many thanks for the response.

Yes, it is weird, since it seems that v4l2loopback is correctly working. It seems more a problem of opencv or gstream, I am going to post it in general opencv forums to see if there is any hint. We have been using version 3.4 for a while and it was working quite well so we kept on that. We are considering an update but for the short term we need to work with this version. We will definitely try your suggestion and we will come back here with any update.

Thanks again

Hi, OpenCV 4.0 works nice. I don’t know why it worked for me before with OpenCV 3.4…but problem solved.

Thanks for the support!

1 Like