360 Live Streaming from Bicycle Using Satellite - Is my crazy idea possible?

Kenan · July 26, 2023, 4:00pm

Hello developers,

I am humbled to be here, from the posts I’ve read and the YouTube videos I’ve watched, you guys are geniuses. If anyone on earth knows how to make my idea a reality, it’s this community.

A few months ago I decided I wanted to ride my bicycle from the southernmost city in South America to the northernmost city in North America. Somewhat logistically complicated but this is done every year by many people, so I know this is possible. Then, I had the idea to livestream the journey. This has not yet been done and adds a significant challenge, internet access. Luckily, Starlink has recently launched a global roaming plan for their in-motion satellite dishes. I plan to tow this specific dish behind me along with some lightweight lithium ion batteries on a bike trailer.

Now, with internet provided, I needed to decide on a camera. My first instinct was GoPro, and as I looked into their cameras I became fascinated with the 360 functionality. Unfortunately, a phone call with GoPro support confirmed their cameras do not offer live-streaming capabilities. Next, I looked to a popular company called Insta360. Their cameras seemed very professional and well suited to my outdoor adventure, and they do offer some live-streaming capabilities, but they are very basic and lacking. I can’t connect their cameras to a laptop which really limits my ability to process and stream the video input. You can only capture video from an app on your phone.

Finally, I stumbled across Ricoh Theta. The open source nature seems necessary for a project like mine, and seeing the high level of user access and control gives me a lot of hope. In my wildest imagination, I could stream a view from the Theta (mounted to my bike with a stick) that I can pan around using a joystick mounted to my handlebar. So I could freely move the view from myself on the bike to the view in front of me or on either side. I’m thinking I could connect a joystick to a laptop via usb and map that input to a software that controls the camera view. Then I could capture that frame and stream to the platform of my choice. Using a software like OBS I’d also like to add an overlay, displaying to my viewers information such as my speed, location, elevation, and maybe even my heart rate as monitored by an android fitness watch.

I’m slightly worried that I’ve gone crazy. This is so exciting to me and I will do everything in my power to make it happen. But before I get too attached, I’d like to hear if some professionals in the field think it’s even possible. Thank you so much for your time reading this and any input you offer.

Kenan

craig · July 26, 2023, 4:17pm

The camera is not designed for 24/7 live outdoor streaming. You may encounter problems with the camera overheating. This problem will be worse if you have the camera in a clear case in direct sun.

A more feasible solution would be to use interval shooting of still 360 images and push the images up to a server for your audience. The fastest interval shots will be in the 3 to 7 second range per shot, depending on the model.

You could provide streaming at certain times, say an hour at a time.

If you stream to YouTube, I believe there is no way to control the view that is displayed to the audience. I don’t think you can use a joystick at the source to rotate the scene for the viewers.

Some other services may allow control of the view from the source.

My main concern is that the THETA camera line is not designed as an action cam. The GoPro Hero (non-360) type of action cams are going to be more robust on the back of a bicycle on an extended trip outdoor with some moisture and dust.

My second concern is that the camera gets hot during streaming because it needs to stitch the two lenses into a single 4K frame @ 30fps. In winter, it may be fine, but on hot days outside in the sun, the camera may overheat during a long streaming session.

My third concern is the bandwidth requirement to stream 4K video.

Feel free to ask more questions.

Kenan · July 26, 2023, 4:55pm

Craig,

Thank you so much for your time and response. From what I can understand you’ve identified three concerns. The potential for dust and moisture to affect the camera, the camera overheating, and bandwidth requirements for the stream. These are great points you bring up, but I’d like to suggest some solutions and see if you think they would do anything to alleviate the potential problems.

Could the Ricoh clear hard case protect the camera from dust, moisture, and and impact resulting from a tumble on the bike? For the overheating problem, this might seem extreme, but could a determined person create some sort of cooling system using fans or circulated water to reduce the camera’s temperature during operation? According to Starlink, roam customers can expect to achieve download speeds of 5-50 mbps, and an upload speeds of 2-10 mbps. From my brief look into streaming requirements, this seems like it could be adequate. Maybe I could stream at a quality lower than 4k?

As far as the joystick controlling the view displayed on my stream, I had in mind this YouTube video which includes who I believe is you panning the camera around in the GoPro VR Player using your mouse to click and drag. My imagination thought it could be possible to control this movement using a joystick rather than a mouse, and then feeding the GoPro VR player frame to OBS for further processing.

I understand these ideas are pretty whimsical and I will likely not be able to achieve what I’m envisioning, but I want to pursue every avenue of making it a reality before making concessions and settling for a standard action shot from something like a GoPro Hero.

Thanks again,

Kenan

craig · July 26, 2023, 6:06pm

It would protect the camera, but it may introduce glare if the case in direct sunlight. It will also introduce more heat. You could look at the a used THETA V (for lower cost) with the TH-2 case or a cheaper knockoff.

Case information

You’d need to drill holes in the bottom of the case for a fan air input hose and then another hole for the exhaust air.

If you don’t use a case, you can just put the fan directly below the camera.

It should work at 4K. You also stream at 2K if you need to. However, the resolution will likely grainy because the audience does not see the entire 2K scene. They’ll see about SD from a 2K equirectangular.

The limitation for the 360 stream navigation is the YouTube viewer software for a 360 video live stream. In the example above, the camera is streaming directly to another computer using WiFi and the person looking at the video is navigating the scene with a mouse. The camera 360 stream is not going to YouTube.

You may be able to find another streaming service that allows the broadcaster to control the navigation. If you use YouTube, the person viewing the scene will need to control the navigation. At least, I think this is how YouTube works.

If you want to stream a non-360 portion of a 360 stream, the technique you describe above with GoPro View viewer could work.

You can divide the problems into seems feasible, difficult, impossible

problem	difficulty
heat	difficult: you would need to test prior to your trip and likely need some type of fan to get air on the camera
dust and moisture	difficult: you would need a case, which increases the heat, as described above. It seems feasible to run air into the case through a hose. But it still might get hot
video navigation	feasible, but untested. if you use something like OBS to get a non-360 screen capture from a laptop to YouTube, then it may work.

Kenan · July 26, 2023, 6:25pm

Craig,

I can’t express to you how grateful I am for you lending me your expertise. You have really done more than can be expected of someone to help a stranger with a project. Thank you for identifying the main obstacles I will encounter and offering potential solutions. It sounds like although it will be a challenge, what I’m trying to accomplish is actually possible. This is very exciting for me.

I’ll be staying in Punta Arenas, Chile for a week configuring the Starlink dish and camera set up. I will most certainly be returning with more questions when it comes time to actually put this into practice. Thank you again, and I will be giving credit to you and this community in the descriptions of any content I create.

Kenan

biviel · July 26, 2023, 7:33pm

Hi @Kenan ,
Interesting post, I spent LOT of time and effort into live streaming with Z1. Im still working on a solution that may help you, but clearly I thing Youtube also has some timie limitations for live streaming. Im working on a solution to decrease fps when z1 starts overheating etc. I suggest you to test, test and TEST… you can try my plugin, but really I see that heat may be a problem for sure, and bandwidth/quality. Is there a guaranteed minimum upload speed in the contract details? Craig provided really detailed feedback to you covering all possible gaps. You could also consider shorter streaming periods during the day that may work…

Kenan · July 26, 2023, 9:28pm

Hi Biviel,

Thank you for your response and your willingness to help. I will gladly take advantage of your knowledge and experience live streaming with the Z1. Here is a screenshot from Starlink’s website explaining the expected upload speeds. I will be using the mobile priority plan.

I don’t expect perfect, uninterrupted coverage and accept there may be times when the stream is choppy or experiencing connection issues. Hopefully the majority of the stream will be clear and smooth.

The main obstacle I’m now foreseeing is heat. I’m going to attempt to engineer a solution by blowing cool air into the hard case. This mini cooling compressor looks promising. It might provide enough air to cool my batteries and laptop as well. If I need to settle for shorter streams to allow time for the camera to cool off, I will do that. I will be back in these forums discussing my project and seeking help when it’s time to test in early October. Thanks again!

Kenan

biviel · July 27, 2023, 9:11am

Hi @Kenan ,
Now what I was doing 2 month back:
Used my mobile to share cellular 4g+(HSDPA) internet as hotspot and moving around with car and analyzing netwrok changes, optimizing my live streaming plugin. The base of the requirement is adaptive bitrate functionality, but I realized that it could work much better. So when available upload bandwidth decreases, in my plugin I measure LOT of things and lowering video streaming quality dynamically, trying also estimate bandwidth for next 2-4 seconds with some custom algorithms. It may help you too for sure. This functionality isn’t there yet in my plugin, still requires some more tests and work to get there.

Second feature that I’m working on is, to be able to stream to my own flow.tours platform with another protocol “SRT” instead of RTMP. This will again add better stability and quality, also H.265 encoding will work out of Theta Z1 toward my platform and I can setup to restream it to your YouTube channel in theory.
How close will be your car following you, or you are planning to attach that Starlink device to your bicycle? So there will be a WiFi router and Theta Z1 would be attached to your bicycle or helmet?
To have a good signal planning to place a high gain WiFi omnidirectional antenna to top of the car? Outdoor 2.4ghz may work better and on longer distance for sure, in cities they may still be interferences so need to keep close to your car.

Without anti shake/stabilization there will be quality issues too for sure.
Here I was testing a DJI Osmo 4 SE to stabilize while walking around etc.

Stabilization/Anti Shake

There are some devices with software stabilization, BUT I think during live stream those are not looking soo good as in commercials. how are you planning to attach the device to your bicycle?

For 360 cams I think a 2 axis stabilization would help a lot, but I didn’t see any non electric simple stabs that could work.

Heat

I spent lot of time on this and I think I was able to tune my Z1 to it’s limit. I’ve a device with heatsinks, where I used special glues to attach small heatsinks around it’s body permanently, this combined with active coolers can work in a 30-32°C , but a thin white coloring to it’s body would be also important if there is direct sunshine. Here are some non professional shots from my experiments:

Heatsink together with those coolers will do the job, I used a potentiometer to be able to adjust the rotation speed. All this would require work on your end clearly, regarding white color I would use a spray for sure and to make it thin layer of color.

Before doing anything I suggest you to try and stream where you are now and see. You can go to a good spot, where there is good perception and ride and stream through mobile device or Starlink.

Just to highlight that these mods with heatsink will break the warranty for sure and you should be aware of before doing such changes to your devices! Clearly what you are planning to do is beyond regular usage.

I did not read your initial post and the information you shared already:

Questions:

So at the end, you do not want viewers to be able to see the full 360° view?

if stick is too long there will be quality issues and still without stabilization it may provide bad experience to viewers. A simple thing that could improve overall experience: https://img.fruugo.com/product/7/94/807039947_max.jpg clearly, with customizations… Keeping the orientation/direction would be important and to be able to attach to a stick directly, the holder of balancers should be redesigned too.

While riding the bicycle how will you control that joystick? All this during live streaming may just not work easily.

With my plugin it will be possible to put a nadir to the bottom of live stream in camera directly a custom image. For adding overlay, etc. you have to be able to stream first of all wirelessly from 360 camera on your bicycle to the laptop/pc in the car. With current plugin I have this isn’t possible, but I’m working on a plugin upgrade that should work, you should be able to stream wirelessly through WiFi router to the laptop/PC and use the live 360 image as video source and add layers, etc. in OBS studio or vMix.

Exciting, indeed, BUT lot of effort to get there. Test first ASAP live streaming with a Theta Z1 from your bicycle using that stick and you will see more challenges…

craig · July 27, 2023, 1:05pm

If you use the Z1, the case is the TH-3, not the TH-2.

The TH-2 is too narrow for the Z1. The TH-3 already has a hole in the base and another on the side.

Kenan · July 28, 2023, 1:42am

Biviel,

First, thank you for taking an interest in my project and taking the time to offer support and assistance. What I am attempting would be impossible without an active, open development community such as this. I am fortunate to have your input. Your plugin sounds very interesting and useful for my project. The adaptive, on the fly settings updates would be perfect in my situation of constantly changing upload speeds. I have a limited understanding of networking theory and protocols but I understand how SRT would be useful in this application. Thank you for the work you are putting into developing this plugin.

I plan to pull a trailer behind my bike with two 12V 200aH LiFEPo batteries, an Asus Zephyrus G14 laptop, the Starlink satellite dish, router, and the portable cooling device to provide cool air to the camera and trailer. I’ve put some thought into the construction and organization of the trailer, and hope to have a watertight seal when closed and a small air outlet hole in the bottom of the trailer to allow for constant air flow. I’m thinking the air compressor could supply air to the inside of a trailer, and a long hose could connect from the trailer to the camera out in front of my bike. The hose would blow air into the sealed hard case, and another small air outlet hole on the bottom of the case could allow for air flow. So there would be a constant stream of air out of both air outlet holes, and a constant stream of cool air to the inside of the trailer / hard camera case.

Stabilization is a great point and something I should consider. Along with the air hose from the trailer to the camera, I could run a cable to connect the camera to my laptop and provide power to a gimbal. Do you think a constant supply of cold air will be enough to cool the camera, or should I also apply heat sinks? I don’t know a lot about heat sinks but I can research them. I feel comfortable breaking the warranty, Ricoh should not be responsible for anything that happens to this camera in such an extreme use scenario.

Correct, I’d like to be able to pan around the full 360° to show my viewers a frame of my choice.

I’m thinking if I mount this joystick to an area easily accessed by my thumb or pointer finger, I could control the camera view while riding with little effort.

To me, the effort is part of the fun! I have no delusions of this project being easy, or fast, or cheap. Luckily, I have a wealth of time to invest in testing. I am still in the early stages of conceptualization and planning. I will be vacationing in Brazil for the month of September while I finish the rest of the planning stage. Then, I will fly to Argentina and ride my bike to Punta Arenas, Chile. This is the southernmost city where Starlink is available. This will be my home base for testing. Here, I will assemble the entire system and spend time troubleshooting problems. I have as much time as it takes to get sorted, so if it’s possible, with your help, I will get it done.

Kenan · July 28, 2023, 1:47am

Thanks for the info! Do you think maybe on a clear day with little dust or moisture, I could get away with streaming with no case? I’d like for the image to be as clear as possible for viewers especially when riding through beautiful scenery.

biviel · July 28, 2023, 9:08am

I think you just have to test yourself and try before you start the project. Z1 placed inside TH-3 will behave different than I’ve tested, I never tested live streaming with TH-3 so I do not have any data that could help. It will make things worse regarding heating for sure.

My plugin I developed is streaming wirelessly to any router 2.4ghz or 5ghz, but via USB cable it’s not working. Everything is done in camera in this case, encoding, packetizing and transferring to destination RTMP or SRT server (YouTube or any other platform), or after plugin update is done, it can also wirelessly stream to any laptop/desktop running OBS of vMIX in same local network.
About image quality that you asked, the advantage of plugin is that it can stream in HDR mode, which looks much better compared to normal preview quality. This HDR mode preview is at the moment only available through my plugin, when USB cable is used and Z1 used through Ricohs driver, no HDR live stream is operational, which means image will not be so sharp, colors will not be so bright, etc.

For your use case it may happen that streaming through USB cable is a better choice as you will take the whole equipment with you, laptop, batteries, etc. but image quality will be lower than the sample I shared with you, just to be aware of. Start playing and testing ASAP, that’s the best advise I can give you now.

Just another question: why not putting a non 360 action camera to your helmet with a thin USB cable or HDMI going toward laptop at the back, you wouldn’t need a joystick. Clearly there may be other issues. There are water proof models too, but probably not when cable is plugged in, so while charging you would need to make it waterproof. I could imagine using some special silicon based glues to put around the cable plug to fix it and make it waterproof. There are models in theory with inbuilt stabilization too…

craig · July 28, 2023, 12:29pm

My personal opinion is that it would be okay as long as you don’t drop the bike or the camera is mounted in a way to prevent the lens from hitting the ground. The lens sticks out and is prone to damage. There is no cheap way to replace a cracked lens.