HDR plugin to automatically create EXR file. For VFX use

I’ll try to install it somewhere this week and will let you know how things go!

Thanks, Ton

1 Like

I received this idea from an engineer familiar with the RICOH THETA. It seems like a reasonable idea to try. If we modify the plug-in to save the EXR file with the name R05061009_EXR_CHANGE_EXTENSION.JPG, the user of the plug-in can then download the file and change the name to R050061009.JPG

"If this is the case, the reason is that it is not registered because of the unsupported extension.
As a countermeasure, if you output the extension of EXR file as JPG, it will be visible in File Manager, so I think it is better to change the extension to EXR after pulling it on Windows PC. "

Enable File Extension View on Windows 10

To use this technique on Windows, you’ll need to disable the “Hide File Extension Option” on Windows File Explorer.

Reboot THETA V

In order to view camera files with Windows 10 file explorer or Mac Image Capture, I needed to reboot the camera by pressing and holding down the power button for 8 seconds.

Copy Files from THETA to Computer

Drag and drop the files using Windows File Explorer or Mac Image Capture.

Rename File

Change the extension by right-clicking on the file

Test File Integrity

Open in an application like Luminance HDR that lets you inspect and edit the file.

In Luminance HDR

Delete Files from Camera

To save on storage space, delete files from camera.


If you want to test this file, here is an apk of the modified version. If you use this apk, remember to set the storage and camera permissions using something like Vysor.


Sleep Mode

As the plug-in takes 2-3 minutes to take and process an EXR file, you should prevent the camera from going into Sleep Mode too quickly. Using the official RICOH mobile app, you can set the sleep mode higher by accessing the camera settings.

1 Like

It works on a Mac, too. It appears that it’s important to reboot after taking a round of bracketed images. I can copy and delete using Image Capture.

Very nice.

It didn’t work on a Z1 in my initial test. Takes one picture, then stops.

Testing Update May 8, 2019

I did some additional testing on the original plug-in that saves the EXR file with an EXR extension. Unfortunately, the file is not appearing in Windows File Explorer even after rebooting the camera.

At the current time, the plug-in workflow is only usable if the EXR file is temporarily saved with a JPG extension. They will need to reboot the camera to see the files in Mac Image Capture or Windows File Explorer. The plug-in user can download the file to their computer. Once on their computer, they will need to change the filename extension from .JPG to .EXR.

1 Like

Hi all,

i’ve uploaded a new version to github:

  • changed the name everywhere to hdr2exr, be aware this will create a new app on the device which you will have to give permissions again.
  • added the exr named as a jpg hack, let’s see if this works! (it will copy the exr so both should be there)
  • added a test with dynamically changing the stop jumps based on the first picture, hopefully this will generate less over exposed pictures. Number are still a bit random so this may need a bit of fine tuning.

Let me know what you think and if everything works, there should be a apk build in the github. If all is okey let’s submit to the store!

Greetings Kasper

2 Likes

@Kasper,

Really impressed at all the good work you’re doing here. Nice improvements. I tested on both Mac and Windows. The “removethis.JPG” hack works great on both Mac and Windows.

On a Mac, using Image Capture:

On Windows, using File Manager:

probably need to rename the EXR file unfortunately and not make a copy.

Hee Ton,

Did you get the z1 in holland or did you order it from abroad? As soon as I get my hand on it i will try to support it.

Aan vr, mei 3, 2019 om 15:19, Ton Habraken via THETA 360 Developer <> schreef:

1 Like

Ha Kasper,

Ordered it at Cameranu.nl :slight_smile: … If you’d like to try with mine, let me know, I’ve not planned a shoot next couple of weeks…

Cheers, Ton

Hee Ton,

Ik dacht dat ze nog niet te krijgen waren in Nederland omdat er steeds staat dat ze tijdelijk uitverkocht zijn. Maar ze krijgen dus waarschijnlijk steeds maar een paar binnen. Dan ga ik hem gewoon bestellen.

Thanx voor het aanbod voor het lenen maar ben nu toch even druk dus wacht wel even tot we er een hebben op de academie.

Thanx!

Aan do, mei 16, 2019 om 10:04, Ton Habraken via THETA 360 Developer <> schreef:

HDR2EXR Plug-in Now available in the THETA Store

Everyone can download and install!

Developer mode not needed.

https://pluginstore.theta360.com/plugins/com.kasper.hdr2exr/

Description

HDR2EXR is a plug-in for RICOH THETA V to help visual special effects professionally easily grab lighting conditions on-set. Just by pressing the shutter button once, the HDR2EXR plug-in will take 11 bracketed pictures and merge them to one HDR file. The main use is intended for on-set VFX HDR capture.

Details

HDR2EXR measures on-set lighting conditions and bases bracketing on that specific lighting situation. It first takes 1 auto exposed picture to determine basic exposure settings. Based on that information, it sets the lowest ISO and then starts taking pictures.

For each bracket it takes 3 pictures and averages them together to reduce noise. It takes 11 brackets with a 2.5 stop increase. This should be enough to capture almost every lighting situation. Unfortunately, very bright light (like the sun) is still visible with lowest shutter times and ISO. This maybe can be fixed on the new RICOH THETA Z1 with higher aperture settings. This version of the HDR2EXR plug-in does not work with the Z1.

Finally, HDR2EXR automatically merges those 11 pictures into one EXR file ready to be used in NUKE, MAYA, etc. Creation of the EXR file is done by utilizing OpenCV HDR libraries.

Installation

And make sure to set the plugin as the default plugin to use.

How to use?

Start the plug-in by holding down the Mode button for 2 seconds. The little LED will turn white. And the Wi-Fi LED will turn Magenta.
Place the camera in the chosen location (use a tripod, shooting handheld will lead to crappy pictures) and push the shutter button. You have 5 seconds to run away and hide, else you are in the picture.
The Wi-Fi LED will turn green and the THETA will make picture taking sounds. It takes 34 pictures (1 to measure lighting and 3 times 11 brackets).
After the picture taking the Wi-Fi LED will blink red and blue. You can now move or pick up the camera. It is busy merging the pictures. This takes about 1-2 minutes. When it is done it will make a sound and the Wi-Fi LED will turn Magenta again.
Connect the THETA to a computer to download the pictures.

Good to Know

After running the plug-in, reboot your camera. The new files will not show up properly on Windows or Mac without rebooting.
In order to be copied and moved over to Windows, the EXR file is first created with a .JPG extension. It comes in this format: “EXRFILER05071408_removethis.JPG”. On Windows, using File Manager, copy over to your computer and then change the extension to .EXR. On a Mac, use the app Image Capture. Both the EXR file and the “EXRFILER05071408_removethis.JPG” will be available to copy over.
The default sleep setting for THETA is often 3 minutes. You should use the RICOH THETA mobile app to set sleep to OFF.
HDR2EXR tries to keep the ISO as low as possible but also the exposure time. When exposure gets above 1 sec, it increases the ISO until it runs out of ISO and then increases exposure time again. :wink:
This version works with OpenCV 3.4.4. I ran into to some problems with 4.0 which I couldn’t fix right away. It also generates a tonemapped jpg, just for fun. I haven’t been able to get this jpg to show up in the THETA iOS app. Don’t know why, maybe someone can help?
If you want to build it for yourself make sure to change the file paths in the Android.mk file (in the app folder).

Credits

The picture taking part is largely based on the work of Ichi Hirota’s dual-fisheye plug-in https://github.com/theta360developers/original-dual-fisheye-plugin
The integration of OpenCV is a combination of RICOH Blog Post: "Running OpenCV in Your RICOH THETA" and https://www.learn2crack.com/2016/03/setup-opencv-sdk-android-studio.html and a lot of trail and error!
The HDR part is based on https://www.learnopencv.com/high-dynamic-range-hdr-imaging-using-opencv-cpp-python/
Feel free to change, improve and of course use!
Let me know what you think and run into!

4 Likes

Yo,

The following is a post of unfortunately limited utility, but just to offer a round of applause for building such a useful tool.

I’m a 3D artist drawn to the theta for making speedy HDR maps on set for post production use. With the new Z1 hardware, what’s not to love?.. Well, the accompanying software. The promise (or perhaps I misunderstood) of easy HDR creation at the click of a button is tantalising, but given the current SW experience it’s a promise that isn’t delivered.

Awkward tooling, poor documentation (my favourite thus far has been a reference “See the help”… in the help), and a few errors that I’m currently unable to deal with, and I cannot at the time of writing take my exposure bracketed snaps and transform them into something useful. Hugely, hugely frustrating. Again, great hardware on paper, and the potential for a super-useful solution to HDR creation on a budget, but I just don’t feel as though the current SW experience has my back.

HOWEVER! The existence of this tool fills me with hope that the answer to my current nightmare may one day be realised.

Thanks again,

2 Likes

@FridayMarch26th, thank for the note of hope.

Please note that the plug-in does not currently work on the Z1. I think that only a minor modification is needed to the threading to the larger Z1 image sizes. Do you have a THETA V or a Z1?

If you have a THETA V and use this plug-in, make sure you understand that you need to change the extension from *.JPG to *.EXR on your local computer.

If you have any questions about the plug-in, please post them here.

Let’s keep the hope alive!

I’m a Z1 owner (…of around two hours), and yes, I note that the Z1 isn’t yet supported by this tool.

Really, I just wanted to note somewhere that the current Z1 software is not good, while sounding up beat about the future of the product.

So fingers crossed for that future. Let’s hope that as time passes development continues and the SW catches up with the HW.

2 Likes

@FridayMarch26th, thanks for the note. It did sound upbeat and hopeful. :slight_smile:

I can provide a little assistance with the following:

  • if you want to provide more specifics on the limitations of the Z1 software, I can pass the information to the product manager. In particular, are you referring to the workflow of using the mobile app to take bracketed shots (how many?) and then using Photoshop / Lightroom (or maybe specialized software) to create the HDR file. From there, do you put it into something like MAYA?
  • as Kasper was nice enough to open source the software, I was able to test it on the Z1. @jcasman and I may be able to help Kasper figure out where the problem is. I think the problem is minor. If Kasper fixes the problem, I can definitely help him to load an updated plug-in to the store

Additional note. I only looked at the problem for 10 minutes, but I think it may be related to the larger Z1 images taking up more processing cycles and I think it may just require the “merging” process to be run on a background thread (which is fairly simple to implement). I’m hopeful that it’s an easy fix. :-0

@FridayMarch26th, thanks for the feedback and all the details. Can you point out this self-referencing help link that you’re referring to? Is this in the RICOH THETA mobile app? The HDR2EXR documentation? Where did it come up?

Will it work with version 3.00.1 firmware?
I was able to shoot, but removethis.JPG is not generated.

The reverse of this technique could work for the EXR file.

Here’s a skeleton to run an HTTP server inside of the camera. It would need to be adapted for the specific file format of EXR.

Here’s an article and longer example of using a HTTP server that runs inside the camera.

There may be a problem with firmware 3.00.1. I’m checking it now. The plug-in is stopping with a pi-pi-pi error sound, which is not normal behavior. Asking @jcasman to help with testing now.

I will also test with firmware 2.50.1, which was the last version of firmware tested before submission.

test with firmware 2.50.1

Test 1

result: successful
firmware: 2.50.1
reboot: yes


Test 2 - no reboot

result: stuck on red flashing LED with no blue LED mixed in. No HDR file produced
firmware: 2.50.1
reboot: no

  • took out of plug-in mode
  • set back into plug-in mode
  • took three attempts to put into plug-in mode (came out of plug-in mode after a second on two attempts)

Test 3

result: success
reboot: yes
firmware: 2.50.1

WiFi LED solid magenta

  • 0:50 flashing green and taking pictures
  • 0:50 to flashing red with blue LED blinking occasionally
  • 2:00 blue flashing
  • 2:30 red flashing
  • 2:40 ending sound. One continuous sound. Solid Magenta
  • press lower mode button to take out of plug-in mode


Test 4

test condition: delete all old files from camera. reboot power of camera by holding down power button.
result: successful
firmware: 2.50.1
reboot: yes

Wi-Fi LED is solid magenta at start of test

  • 0- 0:50 - flashing green
  • 0:51- 2:05 flashing red with some blue
  • 2:06 - 2:40 flashing blue
  • 2:41 - 3:00 flashing red
  • 3:01 - end sound, solid magenta

Take out of plug-in mode by pressing lower mode button.
Connect to Windows 10 computer with USB cable.

Testing HDR file with Luminance HDR. File appears to contain all relevant light information.


3.00.1 firmware tests

Note: first three tests failed. Plug-in completes all shots per the procedure above. Does not save HDR file to camera storage. Will stop working with “pi-pi-pi” sound

Error using debugger

2019-05-31 04:26:14.488 2454-2454/com.kasper.hdr2exr I/MainActivity: Done with picture taking, let's start with the HDR merge.
2019-05-31 04:26:14.488 2454-2454/com.kasper.hdr2exr D/MainActivity: times length is: 11
2019-05-31 04:26:14.489 2454-2454/com.kasper.hdr2exr I/MainActivity: Merging average pics for denoise.
2019-05-31 04:26:14.614 2454-2454/? D/AndroidRuntime: Shutting down VM
2019-05-31 04:26:14.629 2454-2461/? I/art: Debugger is no longer active
2019-05-31 04:26:14.629 2454-2461/? I/art: Starting a blocking GC Instrumentation
2019-05-31 04:26:15.175 2454-5696/? W/art: Thread attaching while runtime is shutting down: Binder:2454_3
2019-05-31 04:26:15.175 2454-5696/? I/AndroidRuntime: NOTE: attach of thread 'Binder:2454_3' failed
1 Like

I’ve tested the HDR2EXR plug-in on a THETA V running firmware 2.50.1. This means it’s currently one update behind the 3.00.1 firmware.

The plug-in works well for me in multiple tests over several hours.

  1. Downloaded and installed HDR2EXR from the store using USB
  2. Set it as the boot plug-in using the RICOH THETA desktop app using USB
  3. Set THETA to Plug-in Mode - Main front LED goes from blue to white and the Wi-Fi LED turns magenta (pink)
  4. Shutter button starts the plug-in, it takes 34 images - This takes a little time - Approx 1 minute
  5. When it’s done taking the pictures, while is it busy creating the HDR file, the Wi-Fi LED blinks both blue and red - This also takes a little time - Approx 2 minutes
  6. When the HDR file is created, the Wi-Fi LED returns to magenta (pink) - I did this multiple times in a row - Sometimes I waited several minutes or sometimes I pressed the shutter button again almost immediately - Works fine
  7. Note: You need to reboot the THETA to see your HDR file - This is true for both Mac (using Image Capture or RICOH THETA File Transfer for Mac) and Windows (using File Manger)

Summary: HDR2EXR works for me using 2.50.1 firmware. If you want to use it, I believe it is not “easy to use,” but it does, with just one button press, nicely find the lighting settings and then use those to set the ISO for 11 bracketed images, and then produces an HDR file.

Getting that file off the THETA requires a reboot. And you need to understand that you then pull the file off the camera, rename the extension (format is this: “R05311154.EXR_removethis.JPG”) from .JPG to .HDR in order to get what you want. It’s not a big challenge, but it requires following the steps.

Example from Windows File Manager showing 8 HDR files (“R05311154.EXR_removethis.JPG”) inside the THETA ready to be moved to my computer