Dual-Fisheye Plug-in With Camera Metadata - Template for Developers

codetricity · June 11, 2019, 11:43pm

Thanks to @shrhdk and his theta4j library, I was able to build a dual-fisheye plug-in in under an hour. This is a simple template for developers to customize and finish a plug-in with the features they want.

Source Code

Complete source code for this simple plug-in available on GitHub.

Pre-compiled apk

There is a pre-compiled apk in the release folder. To test this apk, you will need to have a THETA V in developer mode.

Interval Shooting Modification

If you want to set up interval shooting, you can use this example:

github.com

codetricity/scuba/blob/5280b9b6e4bf20845ba9a0433064c872caf1f8c2/app/src/main/java/guide/theta360/scuba/MainActivity.java#L116

    
      
          
          
// https://developers.theta360.com/en/docs/v2.1/api_reference/options/sleep_delay.html
          theta.setOption(SLEEP_DELAY, 65535);
          // https://developers.theta360.com/en/docs/v2.1/api_reference/options/off_delay.html
          theta.setOption(OFF_DELAY, 65535);
          Log.d(TAG, "Sleep Delay: " + theta.getOption(SLEEP_DELAY).toString());
          Log.d(TAG, "Off Delay " + theta.getOption(OFF_DELAY).toString());
          
          

          

          
while (currentPicture < maxPicture) {
              Log.d(TAG, "current picture " + Integer.toString(currentPicture));
              Log.d(TAG, "Color Temperature " + theta.getOption(Options.COLOR_TEMPERATURE).toString());
          
          
    notificationLedBlink(LedTarget.LED3, ledColor, 1000);
              notificationAudioSelf();
              Thread.sleep(delay/2);
              notificationLedBlink(LedTarget.LED3, ledColor, 500);
              notificationAudioSelf();
              Thread.sleep(delay/4);
              notificationLedBlink(LedTarget.LED3, ledColor, 250);

Adjusting Color and Other Photo Options

if you want to adjust options of each picture such as colorTemperature, an example is here:

github.com

codetricity/scuba/blob/5280b9b6e4bf20845ba9a0433064c872caf1f8c2/app/src/main/java/guide/theta360/scuba/MainActivity.java#L152

    
      
                  catch (InterruptedException e) {
                      e.printStackTrace();
                  }
              });
          }
          
          
if (keyCode == KeyReceiver.KEYCODE_MEDIA_RECORD) {
          
          
    colorExecutor.submit(() -> {
                  try {
                      theta.setOption(WHITE_BALANCE, WhiteBalance.COLOR_TEMPERATURE);
                  } catch (IOException e) {
                      e.printStackTrace();
                  }
                  switch (colorTemperature) {
                      case 2500 :
                          colorTemperature = 6500;
                          notificationLed3Show(LedColor.YELLOW);
                          try {
                              theta.setOption(Options.COLOR_TEMPERATURE, 6500);
                          } catch (IOException e) {

theta4j tutorials

HTTP Response from Camera

The response branch shows how to get the camera HTTP response from the WebAPI command. It shows how to receive the URI for the last picture taken.

Get and Set Camera Options

The options branch shows to to get and set camera settings. ISO, exposure compensation, capture mode.

Audio and Visual Indicators of Camera Status

The indicators branch shows use of a camera beep and magenta LED to show the start of a multi-picture loop.

Display Last File to Vysor, Getting URL and Path

The response-vysor branch displays the last file taken to the virtual THETA screen using Vysor. It also outputs the URI and the file path for the last file taken.

jcasman · June 12, 2019, 6:23pm

@codetricity, thanks for building this, it looks really useful. I was not able to take pictures with it yet.

Few notes from my testing:

It seemed like the plug-in would not show up in the list of plug-ins in the RICOH desktop app until after I rebooted the camera after installing the theta4j-fisheye apk with adb. Might be just a glitch on my side, but I noticed it.
Did not have to set permissions using Vysor. Obvious, I guess, since it does not use the camera or storage or other things. But I noticed this, too.
After theta4jfisheye apk is installed and set as the boot plug-in, it switches in to Plug-in Mode, no problem. But when I press the Shutter button, nothing. If I jump out of Plug-in Mode, it takes regular pictures, like it’s supposed to.
I double-checked to make sure it’s properly set as the boot plug-in. It is.

I’m guessing I’m missing something basic. Any hints appreciated.

In the meantime, I’m uninstalling and re-trying. I’ll be back with my results.

Jesse

codetricity · June 12, 2019, 7:54pm

Can you download the apk again and reinstall the plug-in, then reboot the camera by holding the power button down for 18 seconds?

I just changed it to use theta4j 1.3

I can’t replicate the error you reported

Looking at your log files, it seems similar to this problem:

github.com/ricohapi/theta-plugin-sdk

Camera fails to start up from sleep

opened 01:27PM - 03 Jun 19 UTC

TomRaybould

Model: Theta V Firmware: 2.50.1 Over the weekend while one of our developers… was testing our plugin application, a problem occurred where the camera failed to startup from sleep mode (described below). 1) Camera is sleeping. 2) Press power button. 3) Camera wakes up. 4) The error at the bottom is thrown in logcat. 5) The device beeps and blinks red and then goes back to sleep. This seems like pretty major issue seeing as if this happens to our client users it makes the camera totally unusable if they don't know how to fix it. Can you please give me any reasons as to way this error would be thrown by the camera? Solution 1) Press power button to wake up camera. 2) Immediately hold down power button to power off camera. 3) Restart camera. While this works, it is not a viable solution to the problem for all our clients to know how to troubleshoot the cameras. Also I am pretty sure this isn't caused by something we are doing in the plugin as we have been testing it for weeks without issue. Thank you! ERROR 2019-06-03 08:49:48.594 1175-1396/? E/ActivityManager: Sending non-protected broadcast com.theta360.devicelibrary.receiver.WLAN_OFF from system 1896:com.theta360.receptor/1000 pkg com.theta360.receptor java.lang.Throwable at com.android.server.am.ActivityManagerService.checkBroadcastFromSystem(ActivityManagerService.java:18095) at com.android.server.am.ActivityManagerService.broadcastIntentLocked(ActivityManagerService.java:18576) at com.android.server.am.ActivityManagerService.broadcastIntent(ActivityManagerService.java:18758) at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:499) at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2869) at android.os.Binder.execTransact(Binder.java:565)

I’m generating the dual-fisheye images in my tests

codetricity · June 12, 2019, 10:46pm

put the “setting option” and “taking picture” on different threads
have delay between options and action

jcasman · June 12, 2019, 10:59pm

UPDATE: Success!

Did more testing with @codetricity’s new version, posted here: theta4j-fisheye/release at master · codetricity/theta4j-fisheye · GitHub

NOTE: I’ve been working with the compiled apk, not building it from source.

ALSO NOTE: I was able to test it with firmware 2.50.1. It does not work. But, it’s not supposed to work.

It works well with firmware 3.00.1. I am testing on a Windows 10 laptop.

My Steps

Downloaded the apk, used adb install to install it on my THETA
Checked permissions using Vysor. Do not need to set any.
Used the THETA desktop app to “active” the theta4jfisheye plug-in. (In other words, set it as the boot plug-in.)
Trying to jump into Plug-in Mode did not work. Needed to reboot the camera.
Set to Plug-in Mode (white LED) and shutter button takes pictures in dual-fisheye.

It appears there needs to be a delay (I guess very slight) between setting to ImageStitching and then taking a picture.

L34: theta.setOption(IMAGE_STITCHING, ImageStitching.NONE);

L48: theta.takePicture();

There are possibly various ways to accomplish this. But this code template is working fine in my testing.

github.com

codetricity/theta4j-fisheye/blob/381feb3e2980508bd3456b286f250bdaea63eac0/app/src/main/java/org/codecakes/theta4jfisheye/MainActivity.java#L21

    
      
          import org.theta4j.webapi.ImageStitching;
          import org.theta4j.webapi.Theta;
          
          
import java.io.IOException;
          import java.util.concurrent.ExecutorService;
          import java.util.concurrent.Executors;
          
          
import static org.theta4j.webapi.Options.IMAGE_STITCHING;
          
          

          
public class MainActivity extends PluginActivity {
              final Theta theta = Theta.createForPlugin();
              private ExecutorService executor = Executors.newSingleThreadExecutor();
              private ExecutorService pictureExecutor = Executors.newSingleThreadExecutor();
          
          

          
    private KeyCallback keyCallback = new KeyCallback() {
                  @Override
                  public void onKeyDown(int keyCode, KeyEvent keyEvent) {
                      if (keyCode == KeyReceiver.KEYCODE_CAMERA) {
                          executor.submit(() -> {

codetricity · June 12, 2019, 11:26pm

This was the log you sent me. There’s something about a non-protected broadcast that I don’t understand. It appears to work now on your camera.

06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:126)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:147)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:121)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:254)
06-12 12:06:01.991  2290  2966 W System.err: 	at okhttp3.RealCall.execute(RealCall.java:92)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.theta4j.osc.OSCClient.httpPost(OSCClient.java:419)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.theta4j.osc.OSCClient.commandExecute(OSCClient.java:189)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.theta4j.osc.OSCClient.setOptions(OSCClient.java:341)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.theta4j.osc.OSCClient.setOption(OSCClient.java:326)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.theta4j.webapi.Theta.setOption(Theta.java:225)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.codecakes.theta4jfisheye.MainActivity$1.lambda$onKeyDown$0$MainActivity$1(MainActivity.java:32)
06-12 12:06:01.992  2290  2966 W System.err: 	at org.codecakes.theta4jfisheye.-$$Lambda$MainActivity$1$sO2BKa_sW5VyLC8Q5G_KAgF7vtM.run(lambda)
06-12 12:06:01.992  2290  2966 W System.err: 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:428)
06-12 12:06:01.992  2290  2966 W System.err: 	at java.util.concurrent.FutureTask.run(FutureTask.java:237)
06-12 12:06:01.993  2290  2966 W System.err: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
06-12 12:06:01.993  2290  2966 W System.err: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
06-12 12:06:01.993  2290  2966 W System.err: 	at java.lang.Thread.run(Thread.java:761)
06-12 12:06:02.276  1172  1330 D WindowManager: interceptKeyTq keycode=27 interactive=true keyguardActive=false policyFlags=22000000
06-12 12:06:02.277  1172  1329 D WindowManager: interceptKeyTi keyCode=27 down=false repeatCount=0 keyguardOn=false mHomePressed=false canceled=false
06-12 12:06:02.283  1902  1902 W ContextImpl: Calling a method in the system process without a qualified user: android.app.ContextImpl.sendBroadcast:877 android.content.ContextWrapper.sendBroadcast:421 com.theta360.receptor.view.CameraView.onKeyUp:2329 android.view.KeyEvent.dispatch:2715 android.view.View.dispatchKeyEvent:9960 
06-12 12:06:02.285  1172  1830 E ActivityManager: Sending non-protected broadcast com.theta360.plugin.ACTION_KEY_UP from system 1902:com.theta360.receptor/1000 pkg com.theta360.receptor
06-12 12:06:02.285  1172  1830 E ActivityManager: java.lang.Throwable
06-12 12:06:02.285  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.checkBroadcastFromSystem(ActivityManagerService.java:18095)
06-12 12:06:02.285  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.broadcastIntentLocked(ActivityManagerService.java:18576)
06-12 12:06:02.285  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.broadcastIntent(ActivityManagerService.java:18758)
06-12 12:06:02.285  1172  1830 E ActivityManager: 	at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:499)
06-12 12:06:02.285  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2869)
06-12 12:06:02.285  1172  1830 E ActivityManager: 	at android.os.Binder.execTransact(Binder.java:565)
06-12 12:06:02.299  1172  1830 E ActivityManager: Sending non-protected broadcast com.theta360.plugin.ACTION_KEY_UP from system 1902:com.theta360.receptor/1000 pkg com.theta360.receptor
06-12 12:06:02.299  1172  1830 E ActivityManager: java.lang.Throwable
06-12 12:06:02.299  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.checkBroadcastFromSystem(ActivityManagerService.java:18095)
06-12 12:06:02.299  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.broadcastIntentLocked(ActivityManagerService.java:18667)
06-12 12:06:02.299  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.broadcastIntent(ActivityManagerService.java:18758)
06-12 12:06:02.299  1172  1830 E ActivityManager: 	at android.app.ActivityManagerNative.onTransact(ActivityManagerNative.java:499)
06-12 12:06:02.299  1172  1830 E ActivityManager: 	at com.android.server.am.ActivityManagerService.onTransact(ActivityManagerService.java:2869)
06-12 12:06:02.299  1172  1830 E ActivityManager: 	at android.os.Binder.execTransact(Binder.java:565)
06-12 12:06:02.753   577   577 I MSM-irqbalance: Decided to move IRQ5 from CPU3 [P:0] to CPU4 [P:1] (banned)
06-12 12:06:02.754   577   577 I MSM-irqbalance: Decided to move IRQ5 from CPU0 [P:0] to CPU6 [P:1] (banned)
06-12 12:06:02.755   577   577 I MSM-irqbalance: Decided to move IRQ5 from CPU1 [P:0] to CPU5 [P:1] (banned)
06-12 12:06:07.406  1172  1330 D WindowManager: interceptKeyTq keycode=27 interactive=true keyguardActive=false policyFlags=22000000
06-12 12:06:07.406  1172  1329 D PowerManagerService: userActivityNoUpdateLocked: eventTime=833610, event=1, flags=0x0, uid=1000
06-12 12:06:07.407  1172  1329 D WindowManager: interceptKeyTi keyCode=27 down=true repeatCount=0 keyguardOn=false mHomePressed=false canceled=false
06-12 12:06:07.411  1902  1902 W ContextImpl: Calling a method in the system process without a qualified user: android.app.ContextImpl.sendBroadcast:877 android.content.ContextWrapper.sendBroadcast:421 com.theta360.receptor.view.CameraView.onKeyDown:2215 android.view.KeyEvent.dispatch:2691 android.view.View.dispatchKeyEvent:9960 
06-12 12:06:07.413  1172  1182 E ActivityManager: Sending non-protected broadcast com.theta360.plugin.ACTION_KEY_DOWN from system 1902:com.theta360.receptor/1000 pkg com.theta360.receptor
06-12 12:06:07.413  1172  1182 E ActivityManager: java.lang.Throwable
06-12 12:06:07.413  1172  1182 E ActivityManager: 	at com.android.server.am.ActivityManagerService.checkBroadcastFromSystem(ActivityManagerService.java:18095)

It works with the Z1 in DNG mode.

With the Z1 using DNG/RAW images, I can only reduce the time down to 6 seconds. If I try to go lower, it will not take all the images.

This is the branch I’m working on.