Hi everyone,
Is it possible to read RAW frames in real-time from a Ricoh Theta S, over USB, using something like open-cv? Planning to feed these raw frames into a custom video codec, and then transmit over the air using a radio.
Thanks,
Simran
The frames are motionJPEG over USB and in dual-fisheye
Use this
may also be relevant, depending on your app.
Thank you for the information. I am afraid MJPEG or any other compressed format is not what we need. Is there any Ricoh Theta model which can provide raw / uncompressed / PNG frames in real-time? Or would you recommend any other 360 degree / fish-eye camera which provides the same?
The THETA V, X, Z1 can provide 30fps 4K in H.264. If you need sequential shots, the X can take sequential 5K shots at around 2.5 secs per shot and 3 secs for 11K images.
You can also take 8K video to disk, not streaming.
For openCV, most people are using 4K 30fps over USB in equirectangular.
If you need the still images, you’ll likely need to use two cameras or switch between streaming and still image. When you detect the object you want to inspect, you can use the USB API to switch to still image and then take a picture.
If it’s on a robot, there is going to be some delay when you switch it, so you may need to use one camera for detection and another camera for still images. However, this increases the cost and weight.
Most people use a single camera.
Here’s more information on the USB API if you want to assess switching the camera mode from live streaming to still image and then back again.
Thanks for the reply, Craig.
We need raw/uncompressed still images over USB in real time.
“For openCV, most people are using 4K 30fps over USB in equirectangular.” Are these raw or uncompressed images, or is it compressed H.264 video frames?
Just to clarify, we want to replace the H.264 codec by our own custom video codec.
compressed with H.264. The frames need to be decompressed on a separate device like an x86 Linux or Jetson device, which will likely increase latency.
If you use the plug-in technology with a V, X, or Z1, you may be able to achieve something closer to what you are looking for. The THETA S and SC2 do not support plug-in technology.
The V, X, Z1 run a version of Android inside the camera. @biviel has a plug-in that streams over wifi. It may be able to stream without compression.
Thanks! V, X, Z1 do sound interesting. Will explore if I can build my own plug-in to stream over USB without compression. If there any links you could suggest as a starting point, I will be grateful.
Do you need to stream over USB? Can you stream it over Wi-Fi? I’m not sure if you can fully output to the USB port from the plug-in.
As long as we can get RAW frames, WiFi should work. Any idea how much is the FPS of RAW frames over WiFi?
You can get 30fps. I do not think you can get RAW frames. I do not know what the options are for compression.
Other cautions:
- X may overheat with wireless live streaming. Probably better to use the Z1 (which unfortunately is more expensive)
- V is no longer in production
I don’t have too much experience with the compression formats.
It’s possible that the camera may not meet your requirements, especially if you are trying to use uncompressed formats to reduce the latency.
I did see that there is an option to enable uncompressed DNG frames using the camera API.
Is there any way we can obtain a camera temporarily and test it out to see if it can be suitable?
Of the camera models, only the Z1 (most expensive) has the DNG still images. I believe this is only for the still images and not streaming. However, I have not tested any other condition.
The best course of action is to rent the camera. There are likely other services like this.
something like this is $30 for a weekend.