Opencv VideoCapture


#1

I’ve been using OSC 2.1 to get a live-preview stream using Python on OSX. That works. Now I want to display it via OpenCV also using python on OSX. In particular I’d like to use the cv2.VideoCapture(url) call. However this seems to require a URL that supports HTTP GET, whereas OSC requires HTTP POST.

Has anyone got the cv2.VideoCapture api to work with OSC with the Theta V?

Another article on this forum indicates that “THETA UVC Blender” is required. Can someone explain a) if this is relevant to using VideoCapture, and 2) exactly what “THETA UVC Blender” is and why it is required, and what is used on non-osx platforms.

Thank you!


#2

Thank you for reporting on this interesting project.

The livePreview stream is MotionJPEG

https://developers.theta360.com/en/docs/v2.1/api_reference/commands/camera.get_live_preview.html

This is a working example of a plug-in to do the livePreview from Line 534.

As the web server in the plug-in provides the livePreview, you might be able to use the plug-in (it’s in the store), then inspect the HTML see to embed it into the web page.

The FOX SEWER rover code might also be a good starting point.

Note that in both examples, there is some type of relay to make the MotionJPEG stream from camera.getLivePreview usable by a web browser.

You might be able to point cv2.VideoCapture(url) at the FOX SEWER ROVER stream as a starting point for a test. If it works, you can then try and use an internal server on the THETA using plug-in technology.

THETA UVC Blender

This is a driver for Windows. You only need this is the THETA V is connected to Windows with a USB cable. If you’re trying to eliminate a Windows machine, you don’t need this.

Can you use the THETA with a USB cable connected to the machine that your OpenCV application is running on?

Or, do you need the connection to be WiFi?


#3

Hi again. Thanks for those links. Very helpful.

My goal is to stream video wirelessly from the Theta V for an art installation. Using the tips above and from elsewhere, I successfully was able to write a python web server to stream video from the Theta to the HTML5 <video> tag. The code is here: https://github.com/rwoodley/OSCVideoStreamExample

However using livePreview limits you to 8 fps, at least if you want decent resolution: https://developers.theta360.com/en/docs/v2.1/api_reference/options/preview_format.html

I guess the next step is to use the wireless live streaming plug-in? (https://pluginstore.theta360.com/plugins/com.theta360.cloudstreaming/) Can that stream at a higher rate than the livePreview? If so I wonder why livePreview doesn’t support that out of the box.

Again, my goal is to wirelessly stream HD at at least 30 FPS. Has anyone got that to work?

So just wanted to check that before moving forward. Thanks!


#4

I think that the plug-in in the store does wireless live streaming at 4K 30FPS (theoretical maximum).

http://theta360.guide/plugin-user-guide/wireless-stream/youtube/

I do not have the bandwidth on my Internet feed to test it. It may be easier with 2K.

image

This is using RTMP.

There is latency if you stream it from YouTube.

I’ve seen the plug-in below demoed and the resolution and uptime was good. I don’t actually know what service is needed from HoloBuilder to use it. You should try and contact them. Maybe they’ll just let you use it for the art exhibit.

https://pluginstore.theta360.com/plugins/com.holobuilder.jobwatcher/


#5

There is no need to go over the internet I don’t think. I plan to do this all on my LAN.


#6

I believe webrtc traffic is going peer to peer. The solution below requires an external signalling server. I don’t think the actual video traffic goes over the Internet.


Note: the info below is a completely different strategy and probably not relevant to Bob’s project

About SiteStream

Just in case anyone wants to put it on the Internet.


#7

Thank you. Re WebRTC, I presume you’re referring to your article on video conferencing where you have a Theta connect via USB to one laptop, then broadcasting to another laptop over WebRTC. Is that what you had in mind? I guess that is feasible, but it requires an extra computer and since I want to use OpenCV, I’d have to parse the WebRTC in python or C++.

I did install the plug-in and was able to connect to my Theta V over my router, which is nice. But the streaming seems hard-wired to use YouTube/Facebook etc. I want to capture the 30FPS stream myself, wirelessly. Is there any alternative to writing my own plug-in at this point? Has anyone cracked that nut already?

Is there a way to do this with ffmpeg? Can it capture the theta plugin wireless stream?

Thanks again.


#8

At the current time, no one has published a project on this site that provides the functionality you want. The current plug-in is hardwired to use a server such as YouTube/Facebook. The video will need to go out on the Internet.

I don’t think there’s an easy fix right now.

It might be possible to find an existing Android app that can stream the phone camera to motionjpeg. The camera on the THETA will appear like a phone camera to the internal Android OS.

I have not tested this code below.

This one also looks interesting.