Ricoh publishes the Wireless Streaming plugin on the Theta plugin store and releases its code on GitHub under the Apache 2 license. The plugin enables 360 RTMP streaming to a custom server at different resolutions and bit rates. It leverages the rtmp-rtsp-stream-client-java open source project. Plugin code releases are roughly annual, the latest release 1.2.1 introduced support for Theta X.
Unfortunately, Ricoh’s Theta cameras struggle with wireless steaming using this plugin, particularly the Theta X which only lasts 6 minutes streaming 4k before thermal shutdown. Surely some optimisation is possible/necessary to manage power consumption and heat generation.
I would like to understand Ricoh’s intent and ambition for the Wireless Streaming Plugin project. Its GitHub page has no wiki or discussion tabs enabled and it appears to be more a periodic code dump with little attempt to engage developers. Is Ricoh actively maintaining the plugin or seeking to optimise its performance? Would Ricoh welcome and engage with ideas, contributions and pull requests on the code? Would Ricoh adopt bug fixes and features from others based on merit? Would Ricoh open discussion on GitHub, explain the root issue, clarify what can be addressed only in firmware vs plugin code, and set expectations on potential improvements and timelines?
The above would help developers and service providers understand if Theta 360 cameras may be viable for live streaming use cases. Progress would surely also help Ricoh both technically and commercially.
hi, @gthomas ,
Interesting and important topic. As a developer, who put huge effort into my own wireless live streaming plugin I also have some thoughts that I would like to share.
Ricoh let companies, individuals to develop various plugins and run them directly on some Ricoh Theta devices, which is great as it expands functionalities of these devices. But In case a company invests lot of effort into a plugin and planning to make it commercial and get some revenue from the plugin, Ricoh may also come with such functionalities, making parallelly, duplicate work and could in theory prevent that 3rd party company to pay-off invested work and money.
Clearly there is also a competition behind, which also I believe drives the development work, however having Ricoh as potential competitor for some use cases, like wireless live streaming, may cause harm to some companies and individuals like me too.
If there would be a release plan of some functionalities, improvements, a roadmap, could help on this for sure, to avoid also conflicts of interest. Spending 100K USD on some plugin development that will never pay off is waste of resources. Open Source is also great, but it also means “it will never be so efficient as it could” as of now for some use cases… efficiency is very important as cutting edge solutions are required for this great hardware.
Yes its a classic dynamic between a platform vendor with SDK and their developer ecosystem…what features to include in native platform/product vs leave room for ecosystem opportunity. Developers often fill in functionality gaps to meet their own needs or to monetise, and over time platforms/products tend to cannibalize some 3rd party developments.
@jcasman and I met with a manager at RICOH to discuss communication.
There are two points:
clarify concern/problem from developer community
clarify request/suggestion from developer community
Clarify Concern
Currently, @jcasman and I are focused on understanding the concerns of independent developers.
dynamic between a platform vendor with SDK and their developer ecosystem…what features to include in native platform/product vs leave room for ecosystem opportunity. Developers often fill in functionality gaps to meet their own needs or to monetise, and over time platforms/products tend to cannibalize some 3rd party developments.
This most recent post by you makes the concern clearer.
I believe that @jcasman and I should continue to focus on relaying this primary concern from independent developers to RICOH management.
My personal opinion is that RICOH is not trying to monetize the wireless live streaming plug-in. I believe it is a reference implementation to show features of the RICOH THETA cameras.
I believe your concern is that even if RICOH does not plan monetize the wireless live streaming plug-in, the open source plug-in may evolve into something that duplicates your value-add features.
Clarify Request
Can you clarify that your point below is only for the Wireless Live Streaming Plug-in from RICOH? Or, are you referring to a roadmap on firmware enhancements?
I would appreciate more openness regarding
i) Wireless Live Streaming Plug-in roadmap
ii) feedback on specific feature requests
iii) firmware development roadmap, features & times
iv) early release to developers of beta firmware
I have tried to keep my posts on these topics separate, but there is inevitably overlap and underlying connection between them…running the Ricoh Wireless Streaming Plugin on the existing X firmware causes early thermal shutdown. The use of fans to cool partly mitigates device overheating, but to fully address likely needs both firmware and plugin code optimisation. A fan also introduces noise on the internal microphone, which could be mitigated if the firmware supported bluetooth microphones!
Just to be clear, in our case, we do not intend to monetise a plugin, rather we seek a 360 camera from Ricoh which supports 4k wireless live streaming for 30-60mins. A plugin can help provide configuration parameters and smoother UX, but it is just a means to an end. What we really want is RTMP parameter config via API and single-touch streaming. See the Logitech Mevo Start which is a native streaming HD camera.
Could you help clarify some goals for work towards reducing heat? Does reducing power consumption have a big impact? What other goals would you set for reducing heat?
For example, if the developer community, which includes RICOH engineers, were able to reduce power consumption by 10%-30%, would that have a large impact on heat reduction? More importantly, would it be enough to reduce overheating below the thermal threshold?
If you have any opinions on what are good technical goals, such as power consumption reduction targets, for reducing heat?
Hi, I’m not a RICOH plug developer, just a user, so this is just a naive suggestion.
I’d log, via e.g adb shell commands, temperature over time with as parameters device type, resolution, bitrate, audio sample rate and whatever else could be relevant. This would allow finding eventually if at least some set of parameters allow to remain under an upper threshold for a minimum duration, e.g 60min.
If this is not the case then I’d profile the plugin by inspecting what takes resources, e.g CPU cycles, via e.g top, and assuming it would show that it’s the streaming plugin the profile it specifically. I do not do Android development (at most build apk with small modification) so I can’t make suggestion there specifically. The point being that profiling the plugin should allow to see if there is a proper bottleneck and if so, if potentially this could be delegated client side. One such example could be stitching on device delegated to stitching server side or even as a shader client side.
Edit: tried on 0.6k and still overheated after 14min on a Z1.
Hi, im doing heat meadurement for mobths now, using different presets for fps, dhooting mode etc. Every 10 seconds during streaming with my own plugin
@Utopiah , you tried wireless live streaming plugin recent version in low rescand it streamed for 15 minutes only? Are you using 2.10 firmware? What do you mean by 0 6k? Bitrate or resolution?
@Utopiah , what was the environment temperature there? Have you seen my fan/cooler I made? Any way something is worng with this latest firmware. I will test original Ricohs wireless live streaming plugin today as when I tested last time, I tihnk I was running earlier firmware, which I was able to stream with my own plugin for hours… vs 28-30 minutes with latest firmware. Well comparing to 14 minutes that you measures it’s twice as efficient as original plugin, but still my goal was to be able to live stream for 1-2 hours from indoor, from a room temp. ~24°C-25°C… And it was working before with that 2.01 firmware.
@biviel was about maybe 25C too and with the official plugin. My goal is to stream for 8 hours so… 8min in 2k to 14min in 0.6k to 30min with your plugin definitely helps but would still not be enough.
* Version 1.2.2 (2022/09/06): Following settings are added for THETA X 1.20 or later.
・4K (3840 x 2160) 15FPS
・2K (1920 x 1080) 15FPS
・1K (1024 x 576) 15FPS
sorry, just read post.
We don’t have specific goals on heat or power consumption. Rather from a user perspective, we want to be able to stream for a minimum of 30 mins (ideally 60mins) before thermal shutdown. Minimum stream requirements; stream only (not simultaneously record), 24fps, 4k equirectangular to 3rd party rtmp server, no fan cooling. Ideally WiFi connected and battery powered, would accept Ethernet connected and USB-C powered. And we would like to see this with the newest model, the Theta X.
@gthomas ,
as per now I’m working on my plugin to see the effort to make it work on X too. I was able to stream already for 10 minutes and I stopped manually… My issue is that somehow I can’t get to the new camerax.jar provided functions, that is particularly created for Theta X to handle camera. I tried without new classes and I was able to get to camera preview, BUT no stitching was in place, so some functionalities were not working correctly.
For now I see this as the best and fastest option, to achieve your needs in Theta X, so I ask for help to try to resolve this error I get when I run streaming and when I try to use new camera class. I used also original Ricohs wireless live steaming plugin to mimic camera setup. BUT my streaming plugin is using slightly different caemra surface/texture and seems like some of them were not implemented in new xcamera.jar
It would be useful to create a new topic called, “plugin library problems with THETA X” and include the information in the post below as the start of the new topic.
Is this the problem with the X?
unable to get camera preview with stitching
methods setPreviewSurfaceWithFlag and setPreviewSurface are not in camera class
get this error
I/CameraControl: InitCamera 1
D/CameraControl: cameracontrol open():false
W/ow.hdrstreamin: CheckJNI: method to register “setPreviewSurfaceWithFlag” not in the given class. This is slow, consider changing your RegisterNatives calls.
E/ow.hdrstreamin: ----- class ‘Ltheta360/hardware/Camera;’ cl=0x159ca3e8 -----
objectSize=284 (224 from super)
access=0x8008.0001
super=‘java.lang.Class<java.lang.Object>’ (cl=0x0)
vtable (6 entries, 11 in super):
0: void theta360.hardware.Camera.F()
1: void theta360.hardware.Camera.G(android.graphics.SurfaceTexture)
2: void theta360.hardware.Camera.H()
3: void theta360.hardware.Camera.finalize()
4: void theta360.hardware.Camera.setPreviewSurface(android.view.Surface)
5: void theta360.hardware.Camera.startPreview()
It would be ideal, if you can respond in a new topic so that I can more easily reference the question you have about the pluginlibrary from the RICOH GitHub repo.
