Ricoh Plug-in Development Explanation

THETA API Usage plugin
,
1

At CES yesterday, Ricoh announced a new plug-in partner program for the RICOH THETA V. This is exciting news for THETA developers as it opens up new business opportunities for the THETA.

In addition to the announcement, Ricoh presented information about the future vision for the plug-in technology and ecosystem at CES. The theta360.guide team also spoke to key Ricoh staff about the plug-in technology and obtained a developer camera to test the THETA V plugin development process under a special “closed beta” program.

Although details of the closed beta developer program have not been announced, the theta360.guide team can help you get started with a discussion on requirements to join the program. Please leave your comments below or DM @jcasman on this forum.

No Computer or Mobile Phone Needed

The plug-in technology can improve the experience of using 360 images and video for both consumers and businesses. By running a full Android OS inside the camera, developers can create custom image processing and media sharing applications.

Currently, uploading to services such as Facebook and YouTube requires an additional step of moving pictures from the camera to a mobile phone and then uploading to Facebook using the mobile phone. The new plug-in technology can either eliminate the mobile phone step or work with future mobile phone apps to improve the user experience.

Potential for Developers

Here are some concept ideas for developers to think about.

Voice-Activated Picture Taking

Automatic Picture Taking and Distribution Using Facial Recognition

Automatically Detect Faces and Blur Faces to Protect Privacy

Immersive Video

Custom Hardware Projector

Ricoh Plug-in Roadmap

Ricoh also announced tentative plans to develop plug-ins to help improve the THETA V experience for everyone.

Development Process

The plug-ins are developed with a standard Android SDK and can be distributed through a future THETA Marketplace.

Initial Hands-on Review of Development Process

Ricoh provided the theta360.guide team with a developer camera and SDK for testing as part of their closed beta developer program. The SDK comes with a sample plugin. We compiled the sample plug-in from source and modified it. The development process is easy and comfortable for everyone familiar with Android development.

Once we built the Android package, we transferred it to the “developer” THETA V with a USB cable.

With the THETA V in developer mode, it can be accessed with the Android Debugger (adb).

Using Vysor to Test The Plug-in

Ricoh suggested that we use the open source Vysor desktop tool to help test the plug-in.

The tool is great and allowed us to see the THETA V with an emulated screen.

We were also able to set the plug-in permissions using Vysor.

As a test, we created a custom plug-in for the construction industry.

Switching Between Plug-in and Camera Mode

The camera can be switched into plug-in mode by pressing and holding the mode button on the camera.

When in plug-in mode, the front LED will be solid white.

Switching Between Plug-ins

To switch between the plug-ins, we needed to use the WiFi API to set the default plug-in. Although multiple plug-ins can be saved to the camera, only one will be activated when the mode button is pressed. We suspect that the developers will either create a mobile app to manage plug-ins or that Ricoh will build a plug-in manager into their mobile app in the future. Currently, the official Ricoh mobile app will only display the active plug-in.

Camera Buttons

The camera buttons are mapped to Android key codes.

The shutter button is key code 27.

Conclusion

By opening up the camera to third-party plug-ins, Ricoh is giving developers the opportunity to create new businesses that incorporate the THETA V. Using common Android development tools, developers can get up to speed quickly and start building simple plug-ins within a few hours. We were able to build our own plug-in for testing in less than a day. Of course, building a commercial plug-in will take more time and expertise, but the path to get started is clear and easy. Accessing more hardware features seems possible as the THETA V uses a standard Qualcomm Snapdragon 625 MCU.

We can’t wait for Ricoh to make this technology widely available to the developer community. If you want to get updates on the latest information for plug-in development, either drop us a note below or send a DM to @jcasman.

4 Likes
2

Simple test to turn on all LEDs with the camera buttons.

Lots of fun so far learning to access camera buttons, storage, lights, and built-in functions like taking a picture.

2 Likes
3

Oh right, I had almost forgotten about the “Disk Full” LED at the bottom. I have seen that a couple of times when I set it to interval shooting during the holidays.

1 Like
4

In the picture below, both LEDs are white. I have the WiFi LED turn white when I take a picture in plug-in mode.

Note: At the current time, I need to reboot the camera twice after the initial plug-in install. After the initial install, I can switch to the plug-in by simply pressing the lower “mode” button for a few seconds.

1 Like
5

I’m working on my plug-in a little bit every day. I’ll update this thread from time to time.

There are 8 LEDs on the camera. I’m now using LEDs 2, 4, 7.

I’m using buttons 27 (shutter button) and 130 (mode button).

When I first turn the camera into plug-in mode, there’s only 1 LED on, number 2, showing in white.

Pressing the shutter button (27) the first time takes the picture and turns on LED 4, the photo LED, and LED 7, the lower red indicator LED.

In this state, the camera is taking all the images for a single construction floorplan. When the camera is ready to take the next picture, the lower LED is turned off.

Right now, I am turning off the bottom LED by pressing button 130 (mode), but I’ll look into how to check for a camera ready state in the future.

1 Like
6

Mini plug-in hacking session with @jcasman

1 Like
7

Thanks, I learned a lot about manipulating the LEDs and their colors!

1 Like
8

If you want to continue to extend the documentation, here’s my suggestions for how to contribute:

image.png1074Ă—329 60.2 KB

1 Like
9

I got the filename iteration to work. The camera saves each image file as a separate file name inside of a special directory called “DCIM/Construction” on the camera.

Below is a test of the number past 9 rooms.

image.png559Ă—666 53 KB

To transfer the files, I’m using adb now for a test. To copy 13 pictures, 42MB, took 1.9 seconds.

$ adb pull /sdcard/DCIM/Construction
/sdcard/DCIM/Construction/: 13 files p.... 21.2 MB/s (42282449 bytes in 1.906s)
1 Like
10

@jcasman, I wasn’t able to get the video working today, but I did spend more time with the standard Android Camera class. The THETA V uses the older android.hardware.Camera class. Information is here. There’s some additional examples on recording video on the main Android developer site here.

https://developer.android.com/guide/topics/media/camera.html

The standard Android documentation also has a MyFaceDetectionListener. As the THETA plug-in uses the standard Android Camera class, I think it’s important for a potential developer to have prior experience with developing Android apps, preferably an app that used both pictures and videos.

1 Like
11

Hi, i would love to start working with new API. I’m working on huge VR project, and we are using current 2.1 API for Theta V. But many features available via this new API would enrich android part of this project. @jcasman would it be possible to join early access program?

1 Like
12

@felislynx,

Thank you very much for your interest in the early beta program. At this first stage, we need to unlock the V directly, so we are focusing on teams in California.

However, I am sending you access to documentation on the plug-in development environment. You should be able to do a closer evaluation to understand if developing a plug-in meets your needs for your VR project.

Fream looks like it does some really great projects. If you review the documentation and are still interested, let’s talk more details about your idea and your timeframe.

DMing you directly with more information.

Jesse

13

Right now, someone from Ricoh needs to get physical access to the THETA V to enable developer mode. In the future, there should be software to enable developers to do this on their own. Hopefully you can send @jcasman additional information on your project.

Does it use the live streaming capability of the THETA V? Images or video?

1 Like
14

We hope to use both capabilities :slight_smile:

1 Like
15

I sent you a DM with @jcasman on the thread, so we can see if we can overcome the logistics issue of getting the developer mode on your THETA V. Please reply to the DM. It’s in the upper right corner of this interface.

16

I did manage to create one dual-fisheye image at 5792x2896, but I can’t replicate the test. I have the camera lying down on its case in the shot below. that’s why one of the lenses is black.


image.jpg724Ă—357 42.6 KB

17

Quick question for anyone who has started playing with the plugin SDK - If a plugin requires a GUI or some type of user-defined configuration (ie. login credentials), would that require a separate companion app for iOS + Android or can the plugin app’s Android UI somehow be “streamed” to the Ricoh Theta smartphone app?

Maybe a less elegant solution (ie. parsing a text / xml file on the local storage) would suffice for now. I’m sure there will be many plugin use-cases that would be best served by a GUI, but I doubt my developers would be willing to write 3 apps (Android Plugin, iOS companion, Android companion) to solve that. Some type of “streaming” UI would be a neat solution.

18

The most likely use case IMO is that people will write a companion app for iOS/Android.

It is possible to stream a virtual screen of the internal Android OS to another device. For development, we’re using Vysor.

https://www.vysor.io/

The sample code does not cover building a plug-in to mirror the screen to another device, but it is possible to build your own.

19

Yeah, I was thinking that was going to be the case. Hmm, I guess we could handle this on the server-side if I can uniquely identify each Theta. Do you know if the serial number physically printed on the Theta matches the serial number returned by Android’s Build.SERIAL interface? If not, do you know of any other unique identifier that could be matched between the Android OS and the physical device that is user accessible?

Thanks!

20

I’m going to try and ask someone I know at Ricoh to see what the best way to get the ID is. I think we can get the serial number, but I haven’t checked myself yet.