Hi,
I want to develop a platform for 360 live conference where the camera is connected to the phone. I need your guide in the following:
The best camera for live streaming is the Z1.
If your application allows, the most stable connection is with USB to a Linux computer and then retransmit from there. For example, if the camera is on a robot with a good power supply, connect it to the controller board or a dedicated NVIDIA Jetson (even a Raspberry Pi will work for retransmission).
If youāre transmitting direct from the camera using WiFi, you should consider licensing the HDR Wireless Live Streaming Plug-in and use the FlowTours platform by @biviel
There are many options for streaming. If you write down more about what youāre trying to do, we can provide more opinions. For example, is the mobile phone viewer a VR experience or does the person scroll through the scene with their finger?
Hi, Athir,
I think that best is to use all native libraries, as close as possible to GPU. Clearly it depends what is your goal, but for live streaming I would not suggest Unity Engine or React-Native, efficiency matters and also code size and complexityā¦ This is a small device, I found that every single frame matters if you want to push as much as possible in a second and stitch the lens images at the same time, and do some corrections on the fly, gpu/heat optimization, etc. I would stick with android GLES opengl nothing else and clearly in Theta I would pick Plugin development approach.
Depends what you need, if you need a small device, that can stream in a warm room temperature for at least an hour or hours, without additional equipment, Ricoh Theta Z1 is the best, but only with my plugin for now. You can directly plug it via USB C too and reatream using OBS Studio or vMix, etc, but there it will not run in HDR video quality mode, only through Plugin, via WiFi.
I always prefer WiFi because I donāt like cables and I prefer moving from one place to another, and to be able to stream from anywhere, with proper internet, but without heavy supercomputer. That was my goal, when I started to develop the plugin ~2-3 years back, right after I started my platform project.
Iām thankful to @craig and @jcasman for building this community, without them I would never find Theta devices, also I would probably never start developing my platform, at least for my plans the size and lens quality (hdr) was necessary. I think that this is still the best camera for live streaming, clearly not with the original software. There are other devices too, I still own a Qoocam 8k Enterprise, it provides nice quality live stream, but itās so noisy, it has an inbuilt vent/cooler, Iām afraid to use is for more than 20-30 minutes, I could literally cook eggs on itās body, itās soo hot sometimes.
Also there is insta360 pro2, but weight is about 1800gr compared to Z1 , which is ~180gr, price is also 4x, 5x higher and there are some additional costs and limitations I think when someone wants to live stream in high quality using it.
I suggest you to rent 1-2 devices and try to live stream ,but first try to find real life examples, like this: https://www.youtube.com/watch?v=NrKL3jxOM2w&t=2892s it was done using the plugin I developed, directly streaming from San Francisco, it was a podcast a few days back. This was done with the actual version of my plugin that is available in Ricoh store for free, now I will release soon a better, more complex update, including volume boost on the fly, etc.
When you look at devices, which are streaming through phone, in lot of cases seems for me that the phone is doing encoding of the video and resolution is lower than 4k. In this scenario I think a special driver is used in android and 360 camera is used as external camera device.
Try searchg on YouTube for real life live stream samples for various 360 devices: https://www.youtube.com/watch?v=Kv09Y8_IpnU , https://www.youtube.com/watch?v=CQJ1hAKxzBQ&t=576s , search also for insta360 pro or pro2 live stream on youtube to compare.
This is one of my latest test, using the new version of my plugin: https://www.youtube.com/watch?v=yk1aoj3D8cU
I used a phone cellular network shared as hotspot. I used a dji osmo device attached Z1 for stabilization.
There is a reason that you cant find lot of 360 live stream real samples on the web. Network is an issue also and overall device quality. There are some but only with professional additional equipment doneā¦and these samples I shared.
I wonder how would that live stream to youtube work for you with my latest plugin update from your office. You know , when you walked out to that inner garden down through those steps. Stream was interrupted multiple times because of network coverage, but I think now it would be much better.
The latest plug-in is not in the store, right?
Itās really raining here in Palo Alto, CA today.
no, latest plugin update isnāt there yet in store. Iām finalizing, those features on that list will be included. Iām working on plugin to release and after I need to decide how to expose control of those features via flow.tours configration page. Raining there? We got some snow todayā¦ peach blossoming was in progress, so it will freeze, most of itā¦ 
There appears to be significant stuttering problems with the Insta360 One X2 sample. Do you think thatās really how it is?
Your HDR Wireless Live Streaming plug-in offers a significantly better experience. If you integrate your plug-in with the Flow Tours backend system with SRT, it will be a significant leap in user experience.
Well, first 5 minutes were fine on that one x2 sample, the rest was a network, protocol issueā¦ SRT would perform much better there, but no other camera can do SRT in cameraā¦
Any way, I think that 's not the only problem, but resolution and overall image quality too. Mod style Port Adelaide Plaza busk- 360 live stream - 4th of September, 2022 - YouTube here is an older one x used, but for live streaming I think itās the same quality. 2k resolution only at both.
Here is an instra360 pro live stream, clearly it wasnāt streamed via cellular network for sure. Even for my stream I could do 10-20% better quality depending on bandwidth. Insta360 Pro Live Streaming Test (4K) 360Ā° - YouTube pro and pro2 are the same as I know if live streaming in camera directly. Of course if there is additional equipment used, it could go 8k, but thatās a whole different use case.
thanks for sharing that. Thatās single channel audio, right?
Can Flow Tours stream stereo or quad channel audio?
If your system can stream spatial audio, that would be a significant differentiation compared to YouTube live stream.
no spatial yet on flow.tours, I know how to proceed, but having difficulties making implementation in low latency mode. Now Iām pushing that very low latency mode, for ātelepresenceā. It requires whole different approach than the earlier implementation I was working on. It may work, but do not have it there yetā¦ I would love it for sure! 
I would need to understand better how different audio channels get into raw PCM data. Even now to make the microphone boost, I had to play with raw data. Do you have any technical details about how many channels are in Z1 or any code sample where āspatialā data is used? thanks! Itās not rocket science at the end. 
ok, 4 channels, clearlyā¦ Iām thinking.
now after checking some specs, its not rocket science to make spatial audio work. webrtc supports maximum 8 channels, it means we could use 4 and depending on orientation lower some channels, whilc other to making louder. Iām not sure about earplugs support. Also I may have to transcode audio channels separately, there may be some sync issues too at the end. at the moment I donāt know how to send metadata from camera directly as part of the stream.but if I make 4 channels arrive to player, that would solve it already. Iām just not sure how to make the experience work when there are 2 speakers only at viewers endā¦ there are spatial speakers, but quite rare I think, so mapping 4 channels to 2 when playing may provide good enough experience.
Soemthing like this I think I could make work on flow.tours: How To Sync Spatial Audio With 360 VR Video - Voyage Audio here if you look directly toward water splash is the same when you look to the opposite side. Try to play when your back is oriented to water splash second time. There will be no difference if using regular earphone/plugsā¦
There are some itnersting spatial headphones: Best Headphones with Spatial Audio Capability 2022: Apple AirPods Max vs. Sony vs. Corsair vs. Razer ā Rolling Stone it would be cool for sure.
There is an intersting article, not for live streaming, but some custom process to have spatial audio into 360 video even if camera wasnāt supporting it by default. Some steps may be useful once. How To Sync Spatial Audio With 360 VR Video - Voyage Audio
Ideally, you can get funding from a government art or music project or some artist or music group with funding so that it pays for the development time.
This is just a concept video below.
I believe they will need at least stereo audio to realize their concept.
They may have it working???
It kind of seems like it is a live stream?
The people viewing need more than 50Mbps.
Itās also possible that they have a pre-recorded video and use chat with the pre-recorded video at a specific time.
which link are you referring to? I checked and couldnāt find any 360 videos with spatial audio. looks interesting however! thanks for sharing! I see encoder in Z1 for some reason doesnāt handle 4 channels for some reason.
thereās some information about a service called ātalk timeā here. There is no actual stream recording. However, it appears that there is some type of live chat.
Iām looking forward to seeing it in the THETA Plug-in Store and downloading it and trying it out!