WebSocket Server running inside the plugin or firmware

THETA API Usage plugin
, , , ,
1

Hello,

I had a conversation on this forum regarding the possibility to use a websocket server to send Theta camera events in real time to 1 or more websocket clients. Now, I’d like to refine this into a use case, and try to propose whether this would be better implemented inside a plugin or it might be better as a websocket service running inside the Theta firmware?

Additionally, I have looked at the automatic-face-blurring plugin, and it runs a web server. So, it does seem the Theta plugin environment is quite capable of running a processor intensive task.

A few example use cases of where having a websocket service would be advantageous:

Camera availability

  1. Theta camera is turned on with WiFi connectivity via client mode / access point
  2. Application joins the WiFi network
  3. Plugin validates camera information, and delivers error message if there’s an invalid data, or unsupported camera models, firmware version, API level
  4. Application could receive this information via websocket and inform the user on what kind of camera is connected, and to update relevant user interface.
  5. Plugin delivers a websocket message when camera state changes (normally this is queried via the existing /osc/state API)
  6. Application could observe the availability and state changes from the camera
  • A real-time availability status could improve user experience, such as displaying the camera status on an application screen, when we’re conducting an automatic photo capture session.

Monitor the event of photo capture

  1. User press the shutter button on the touchscreen of Ricoh Theta X or via the physical shutter button.
  2. Plugin detects the photo capture event, sends it via websocket messsage
  3. Application received this message and fetch available photos via listFiles command
  4. If the application or the plugin decides it’s a new photo, then the app will proceed to download
  5. The plugin could buffer the URL(s) of images that are new and available for download, and the app will just need to download from this set of URL(s)
  • We can implement this buffering on the application side, but a websocket event message is a cleaner approach, as we could easily differentiate between a photo captured from the application camera.takePicture command to a photo captured via Ricoh Theta’s shutter button. For example, some photos can be taken a day before, and we only want to download a photo that was captured when the camera is turned on at a specific time.

In general, having a websocket either on the plugin or as a service running on the firmware will reduce application complexity, and opens the possibility for advanced application such as teleoperation.

A few use cases where I expect it will take up some memory usage (I don’t expect these to be a lot, but I don’t know the limitations of the plugin):

  • Storing a metadata with the plugin, such as in the case of mapping. Where we need to store extra metadata for each photo (3D coordinates, mapping session identifier, reverse geocoding result, etc.) in memory, before we pass it to client application for long term storage.

  • Opening multiple connections with the websocket server running on the plugin, and polling the HTTP API from each of those connection.

Thanks

1 Like
2

@jessearmandse thank you for bringing up this interesting idea. I’ll pass the information on to some technical people at RICOH.