Dual-fisheye Images With THETA V Plug-in


#61

@BrianKitchen10 I have some new information from a technical guy I know at Ricoh.

  1. PTP can be used in plug-in mode
  2. BLE API can be used in plug-in mode
  3. USB OTG can be used if battery is not an issue

It appears that the USB OTG will not charge the camera, thus, it may not be a good solution.

PTP should work, but we don’t have an example. What happened when you tried to trigger the dual-fisheye image with the USB cable using PTP? Did it not detect the camera? For example, if you were using a Raspberry Pi to test the USB connection, can the Raspberry Pi detect the THETA V?

I’ll do some additional tests myself.


#62

You have done so much! I have had to turn my attention to other projects for the last twoish days and will probably get back to it on Monday. I’m testing using ptpcam from libptp2 and get:

PTP: Device Busy
ERROR: Could not Capture

ptpcam will take pictures when the plugin is not enabled. I can get/delete pictures in plugin mode. This kind of seems like the fisheye plugin is holding onto the camera file descriptor and some the other camera app is attempting to respond to the ptp request but cant open the fd.


#63

Thanks for your help providing this information. I’ll work with @jcasman to send another question to a technical guy we know at Ricoh.

I’m just taking a guess, but the plug-in may need to be modified to release control of the camera resources. The Camera API referred to below is the plug-in Camera API.

https://api.ricoh/docs/theta-plugin-reference/broadcast-intent/#notifying-completion-of-plug-in

Notifying Camera Device Control

When using “com.theta360.plugin.ACTION_MAIN_CAMERA_CLOSE” with Broadcast Intent, the fact that the plug-in is controlling camera resources will be reported to the camera, and the Camera API will be able to be used.

When using “com.theta360.plugin.ACTION_MAIN_CAMERA_OPEN” with Broadcast Intent, the fact that the plug-in has released camera resources will be reported to the camera, and the Camera API will not be able to be used.

Update August 3, 2018

This is a snippet from a conversation we had with a knowledgeable developer.

all APIs are available under plugin mode. But if the plugin uses Camera API, they will not be supported. Under the case of CameraAPI, it will be the similar status as sleep mode. WebAPI cannot be available, BLE and USB APIs behave as sleep mode.

Update August 4, 2018

Question: Is there any way to take a dual-fisheye image using a USB or Ethernet cable?
Response: No, those APIs can be used for capturing standard still or video. To capture a dual fisheye image, the Camera API needs to be used. Therefore, there are some limitations as sleep mode.

The camera api referenced above is the plug-in Camera API.


#64

Thanks for the updates. So it sounds like I can’t take a dual fisheye using a remote capture command.

What about this idea… I turn the camera on and then put it into plugin-mode (can that be done remotely?). In plugin mode it will take a picture right away. It then falls into a never-ending loop which monitors the image database for the presence of any items. I know I can grab and delete an image from the ricoh remotely even with a plugin active. Whenever I need a picture, I will delete whatever picture is in the database. The plugin will then see the database is empty then it takes another picture. I can then grab that image, and wait to delete it until I next want an image.

Does that seem like it has a prayer of working?

Thanks for all your help with this, you have saved me a huge amount of time not following useless trails.


#65

I don’t think you can turn the camera on remotely and then put it into plug-in mode.

If the user can manually put the camera into plug-in mode, you should be able to put the plug-in into the loop you describe and have it check for the image and then take the dual-fisheye. You can then send the image to another source using Wi-Fi, BLE, or a cable.

BTW, I got the Ethernet working with a cheap $14 adapter. However, I think the battery is still draining. I have not done a long-term test yet. I’m also not taking a dual-fisheye picture with this implementation.


#66

Updated Dual-Fisheye apk

There is a new version of the apk, built on 8/6/2018.

It’s available in the Plug-in Application Development Guide

  1. app name should now be DualFisheye Plugin
  2. package name should now be com.hirota41.dualfisheye_plugin
  3. Stability improvement on the latest FW
  4. Reduced interval for 1 shot mode (light blue mode)

This apk is being submitted to the THETA Store, but developers can use it now during the assessment and quality control process. You need to have an unlocked camera, which is free to get unlocked if you join the partner program, which is also free to join right now.


#67

Sorry, bad grammar on my part. I meant that I turn the camera on physically and remotely put it into plug in mode. Even if I can’t do that, if what I described should work then it would just be the initial startup where the user would have to do something manually.

Thanks for the ethernet info. I’m going to buy one now.

Thanks for the info about the updated apk, I will also download that now.

I’ll update when I have success with my idea.


#68

@jcasman, do you know if the camera can be put into plug-in mode with the Wi-Fi API or can it be set to boot automatically into plug-in mode? If you don’t know, can you ask your contact at Ricoh?

I seem to remember a discussion about this, but I don’t remember the answer.

@BrianKitchen10 I sent you the direct link to download the updated apk.


#69

Installed DualFisheye Plugin, now with correct application name.

image

Must set permissions manually using Vysor

image

Tests

Single and triple shots work great.


#70

HDR test with Picturenaut. 3 image test using default settings for new dual-fisheye plug-in

Interface

Log

Generates a HDR image from multiple photographs ....
MKHDRI - Version 1.33.0434  (2005 - 2011)

locale is : English_United States.1252
file 'D:\Documents\plugin-samples\dual-fisheye\sample images\20180806141130.JPG' (8-BIT 5792x2896) opened
file 'D:\Documents\plugin-samples\dual-fisheye\sample images\20180806141129.JPG' (8-BIT 5792x2896) opened
file 'D:\Documents\plugin-samples\dual-fisheye\sample images\20180806141128.JPG' (8-BIT 5792x2896) opened
EV : 1.321928
estimate brightness transfer functions ...
estimate camera curve for channel 1 ...
estimate camera curve for channel 2 ...
estimate camera curve for channel 3 ...
combining images ...
dynamic range : 77.864608
true EV span  : 6.282896
max radiance  : 1.16555
success

Stitching

Converted of picturenaut from tiff to jpg format.
Using PTGui for the stitch test. Note that stitching is not perfect with PTGui and is shown only as a rough test. I’ll repeat the HDR test with Ichi’s stitching library when I receive the updated version.


#71

Please note that I confirmed that the camera cannot be powered by the Ethernet USB OTG adapter. This may limit the usefulness of the Ethernet technique, depending on your application. The camera will stop working when the rechargeable battery runs out.

@jcasman is checking to see if the camera can be put into plug-in mode using an API call or boot setting.


#72

Hi, @codetricity. Thanks a lot for the info of updates.
Deleting old plugin is needed to update the dual-fisheye plugin?


#73

Yes, please delete the old plug-in first. Remember to manually set the permissions.


#74

Testing Ichi’s Stitching Library on Mobile Phone

For this test, I used the following procedure:

  1. took three images using Ichi’s DualFisheye Plug-in
  2. downloaded images to my Windows 10 computer and used Picturenaut to merge the 3 files into a single HDR image
  3. moved merged file back to mobile phone to stitch it with Ichi’s mobile app
  4. moved stitched file back to my Windows 10 computer to view in the Ricoh Desktop application

The above workflow is for the test only. Actual workflow in the field will be different.

image

On the camera, the stitched pictures are stored in the folder MiSphereConverter

image

Stitch looks good, but I need to work on the manual lens calibration.

BTW, the stitching application is still available for Android. It has a watermark on the free demo version. Link to download the demo version is in the plug-in developer guide.


#75

7 Image HDR

Modified Exposure Compensation to use 7 images from with values of: -2.0, -1.3, -0.7, 0.0, 0.7, 1.3, 2.0

Using Picturenaut to combine images

image

Merged Image

Stitched Image

Code


#76

Thanks @codetricity, 5.4k new feature also is rolling out now! I’m very busy.


#77

Do you have more information on the 5.4K video feature?

I do not see the information here:

https://theta360.com/en/info/news/


#78

Here, it is. @codetricity

Theta V resolution increased to 5.4K

SV trusted account is necessary to access the Street View News board. If you have enough time, please check the SV Newsletter in August.

Thanks
Toyo


#79

Hi, @codetricity
Issue again. It happened on shooting with updated Dual-Fisheye plugin bracketing mode.(I used mode buttom press on camera)

issue_dualfish

However, I can’t reproduce the issue now. Please have a look.
Toyo


#80

Thanks for the information. @jcasman and I tested the plug-in yesterday using a 7-shot modification. We were not able to replicate the problem, but we will keep testing this.

We are planning to use the source code of the dual-fisheye plug-in as part of a workshop in San Francisco on August 30 from 6-8pm.