Wireless Live Streaming in HDR

[biviel]

Hi!

Would like to show you my Theta Z1 first of all:

And now the longer story:

I picked this camera because

1. its sensor size and because
2. it can connect to WiFi and
3. according to specs it’s able to livestream via WiFi while
4. charges from external battery.

Unfortunetelly these are not completelly true, at least not like I would love to.

I installed the module and tried testing. Results were disappointing, I think as is that plugin allows only 2k resolution and honsetly it was also breaking a lot,like the stream hangs every 2-3 seconds whichever option I choose on web UI. Also the same happens actually when I see the camera preview on my andorid mobile using Ricoh software…It may be related to preview framerate.

I got the code of Wireless streaming plugin and started to analyze, get in touch with the API, etc. At the beginning my conclusion was:

1. Camera gets extremly hot in about 15 minutes and it turns off regardless of settings
2. Quality of stream is very bad as is.

Anyway to make this short I modified a bit wireless streaming plugin to stream 4k, also added some heatsinks to it’s body. I was also playing with framerates as I wanted to minimize bandwidth required for my needs and I do not need 30 FPS.

I realized if I want to make sure I need a dedicated media server to be able to develop the application I need, so setup one and feeded with Theta Z1, doing more and more tests upcoming weeks. Anyway I realized soemthing interesting: this camera can do HDR live streaming too. I’m referring to RicStillCaptureMultiYuvHdr shooting mode. I was able to make it work, the only issue is that its working without stiching, I mean stiching doesn’t work as it should. BUT the image quality, etc. colors, all is much better there.

Correct sample video to watch is below in a reply.

Any of you have thoughts how to improve/correct that stiching without slowing down extremly? Anyway to alter a bit static stiching of this cam through API?

Thanks in advance!

Regards,
Laszlo

[biviel]

I think that for me RicDynamicStitchingLoad Stitching mode may be the key, but I was not able to find any code samples or reference how to use it?When to set parameter RicDynamicStitchingSave? Need to stop camera between Save and Load, I couldn’t make it work yet. If you have any thoughts please let me know! Thanks in advance!

[craig]

Is there a central location to discuss FlowTours on this site?

In particular, I’m wondering if the FlowTours streaming service allows control of the orientation of the stream that the audience views?

Comment is on this video:

RICOH THETA Live Streaming Tutorial with OBS - YouTube

If FlowTours and your plug-in can allow broadcasters to control the default orientation of a view, then the service can function as a type of tour guide.

If there is more than one camera stream on FlowTours, then the audience can switch between tour guide mode and self-guided mode.

[biviel]

hi, @craig , In my system I’m planning to implement a similar feature. When live streaming to my own platform, there would be a regular 360 live stream playback and in the cornet another AI driven view would be visible too. In particular by default this view would show automatically the face of the presenter or the man closest to the camera, or in theory could use some very simple object detection to focus on. However because of live stream and to minimize latency, this has to be very simple and clear detection, like a special drawing attached to the body of the target, etc. This is not implemented yet, but takes an important place in my plan. The end result would be that broadcasters could even upload their own object detection algorithm or preselect a few of these basic presets.
Also when the viewer clicks on these smaller panes it will switch that view to full screen and clicking again would minimize it to right corner again and the 360 view would become dominant again. Implementation of this will start next…

I was on a longer vacation, reviewing my work and still struggle with some challenges in plugin.

In case you have any more questions, let me know!

Feel free to put a link to the youtube comment, linking to this answer from my end as I may not be able to put my comment there. Thanks!

[craig]

I posted the comment.

What is the youtube channel URL you are using to post comments on the YouTube channel? I would like to whitelist your YouTube account using your YouTube channel URL.

There may be niche that your own service could fill. MimoLive used to support this type of streaming, but no longer does.

RICOH THETA Support for Equirectangular Live Stream

old mimolive feature set with THETA S

text on live stream

Control Centering

Twitter feed inserted into live stream

presentation slide inserted into live stream

[biviel]

hi, @craig ,
My channel URL is: https://www.youtube.com/channel/UC0WDEkhoDJ11-pelEi2wkdw

Interesting, thanks for sharing!

mimolive seems for me it’s a commercial software to download to desktop and use it there, connect Theta S was supported to connect to desktop by cable and to mimolive software app, do you know?

I’m planning to support all 360 cams but clearly, need to do some POC’s once I get there…
My main concept is to eliminate need for additional equipment and achieve something similar but without desktop app/cable requirements. All purely on HTTP and front end of web…

[craig]

@biviel

My channel URL is: https://www.youtube.com/channel/UC0WDEkhoDJ11-pelEi2wkdw

I added the channel to our whitelist for the channel. It seems like there is some bug in the YouTube algorithm that is filtering comments from your geographic area. I would love to have your comments on the channel. I sent a message to YouTube support using the “Send feedback” feature. I’ll see if that makes any difference.

@biviel

mimolive seems for me it’s a commercial software to download to desktop and use it there, connect Theta S was supported to connect to desktop by cable and to mimolive software app, do you know?

yes, it is desktop software. The connection was with the RICOH THETA S and a USB cable. The output from the THETA S was motionjpeg over USB. there was also the option to output HDMI direct from the camera.

It seems like the market conditions are better now for live streaming events:

• more bandwidth available
• COVID restrictions encouraged people to try out online experiences

[biviel]

@craig,
whatever you did, it worked, thank you! I was able to comment today, which I did. I’m kind a lost a bit in summer and frustrated by other obligations toward other projects, which I get money from for now. yes, and being unhappy as I do not work on my real project as I should and I could.

Regarding cooling of Z1, seems that a really slow motion, slow noise cooling system like I built is enough to make z1 lives stream with my plugin, wirelessly, in HDR mode too (which generates more heat) for 3-4 hours at least, when temperature is above 89°F . Did I share this long live stream, which is boring, but long and it was a live stream from our garden, with kids, etc. like a home video: Hungary Summertime Theta Z1 live streaming test - YouTube . Also a few images about the fan/cooling system I built, which is powered by same powerbank as Z1 and during 3.5 hour live stream it dropped only by 15% it’s capacity (powerbank 20000mah) with 2 USB outputs. 1 was used to charge Z1 and other was plugged in the fan system. NAdir logo was rendered also by my plugin during live stream.
Also there is an RTPM on fan, so I can adjust the rotation speed, to make sure it’s not causing too much noise. During test I was experimenting with it too.

[craig]

This is fantastic. Do you have the Z1 controlling the speed of the fan?

The audio sounds good in your test.

[biviel]

@craig

This is fantastic. Do you have the Z1 controlling the speed of the fan?

Thanks, Craig! No, I put there a small potentiometer to control the speed of rotation manually, it’s visible on last image. I tried to use cheap and efficient parts in this cooling system. It’s cost is around 20-25USD + invested work to build it.

[craig]

Have you attempted to use this fan system with the THETA X?

[biviel]

Not yet. As when I checked last time it would take much more time to make my plugin work on it, I’m focusing on Z1 and to finish my plugin first for it. Also body of Theta X is a bit lighter, and seems from different alloy, I’m not sure about how efficiently this cooling would work there. From experiences with Z1, it could make live streaming work for 15-20 minutes on Theta X in same environment, indoor, 24°C (~75°F), where I measured about ~3-5 minutes of live streaming with my Theta X, most.

[craig]

For the Z1, have you considered promoting FlowTours by working with film or multimedia schools in your area?

One idea is to have film school students go to a live event such as an outdoor music festival with the FlowTours rig and work on type of interactive online experience with text chat and possibly other interactive activities things.

[biviel]

good idea, thanks! I still have some work to do on FlowTours back end, which I paused, it’s not ready yet, or just partially, but still has some work to do unfortunately. I may need to find investors to get there or it may take more time, where I feel like will loose.

[craig]

Is there any way for you to make money with just your plug-in if you stream to YouTube, Facebook or some other existing service?

For investment, have you considered approaching other cloud service vendors and proposing a merger with FlowTours for a combination of stock and money?

You can start off by researching the companies that have a plug-in on the RICOH THETA plug-in store. Just click on each company, then go their web stie. Then ask if they want to cooperate technically, with your plug-in to stream to their service with RTMP. This could be a way to start a “general” discussion that might lead to more detailed “financial” discussions naturally.

RICOH THETA Plug-in Store

[biviel]

yes, @craig , that’s what I’m planning to do. I feel in some cases limited, clearly it’s also summer… I complained to you and I feel better already. Thanks for your support!

[craig]

Can the current plug-in you have now reduce the framerate of the stream as a possible way to reduce the heat generated by the camera as well as the bandwidth requirement?

[biviel]

For now, need to set before streaming. I was thinking about to enable a workflow, where the user can set “decrease FPS if cam gets hot and increase again if it cools down”, but not sure if it would be better than just setting the FPS right before streaming. it may not make sense to go down to 15FPS. At 15FPS I know that my plugin is able to stream even if it’s quite hot, for hours wirelessly, without any cooling and in HDR mode. But I was thinking about the more complex algorithm like you also suggested, may be later. I measure temperature during streaming even now… Regarding bandwidth form my tests I can tell it doesn’t make much difference.

[craig]

Is there an interface for the plug-in so that you can adjust the framerate of the video?

This might be useful for the X as a way to reduce the heat of the THETA X Wireless Live Stream.

[biviel]

yes, @craig , these are actual options exposed to each Z1 users: