Possibility of live streaming dual-fisheye images via USB

[idnsunset]

Hi, theta360 community

Our team have a project about route planning based on computer vision and recently switched to RICOH THETA Z1 for extended FOV. Our algorithm was designed for perspective images so we have to convert the equirectangular images to perspective ones in real time. My questions are:

1. Is it possible to have THETA Z1 live stream dual-fisheye images via USB?
2. If not, one of my colleague has ever developed an algorithm that converts fisheye images to perspective images, he thinks it is not that difficult to implement an equirectangular-to-perspective conversion as long as he knows the paramters that THETA built-in stitcher uses to perform the fisheye-to-equirectangular stitching. Has anybody in the forum ever done a similar job like this? Or does anybody know those parameters required to perform the conversion?

Sorry for having no time to view all of previous threads, probably I missed something. Thanks in advance!

[craig]

@idnsunset

Is it possible to have THETA Z1 live stream dual-fisheye images via USB?

Unfortunately, there is no way to do this. The THETA S (old model) is the only model that could stream over the USB cable in dual-fisheye

@idnsunset

If not, one of my colleague has ever developed an algorithm that converts fisheye images to perspective images, he thinks it is not that difficult to implement an equirectangular-to-perspective conversion as long as he knows the paramters that THETA built-in stitcher uses to perform the fisheye-to-equirectangular stitching. Has anybody in the forum ever done a similar job like this? Or does anybody know those parameters required to perform the conversion?

The stitching parameters of the lens are not disclosed. Other people are implementing the equirectangular to perspective without the lens distortion parameters and it appears to be good-enough in some cases.

If it is limited production, you can manually calibrate each camera and save the correction parameters to a settings file of your application.
.

---

Robotics Intelligence Laboratory using THETA S. However, they are also using the THETA V for some prototypes. You may want to contact the OP and ask what the current status is.

Live Streaming over USB on Ubuntu and Linux, NVIDIA Jetson

For people interested in using Ricoh images for applications in ROS and openCV, check the following github pages: 1> https://github.com/RobInLabUJI/ricoh_camera 2> https://github.com/johnny04thomas/rgbd_stereo_360camera Hope this helps.

[biviel]

hi! May I ask you why is USB connection necessary for your use case?Why nor live stream wirelessly in dual fisheye format?

Thanks!

[craig]

@biviel

hi! May I ask you why is USB connection necessary for your use case?Why nor live stream wirelessly in dual fisheye format?

Does your plug-in have an option to allow streaming in dual-fisheye?

[biviel]

hi! Yes, it does work. It’s using WiFi connection and RTMP protocol for now. My guess would be that they need very low latency and with RTMP that may be an issue. I hope will get an answer here soon.

[idnsunset]

We need to work outdoors where wifi is not always available. Another reason is that we are working on Linux and our application was designed to read data from v4l2 video devices.

[idnsunset]

Latency is one of what we are concerned about as we need to process the live streaming images as quickly as possible in real time. We’ve tested the gstreamer pipeline with nvdec, the latency appears to be acceptable when streaming in 2K resolution. Streaming in 4K is intermittently stuck every a few seconds, although our project doesn’t need 4K images indeed.