HDR plugin to automatically create EXR file. For VFX use

@Kasper,

Really impressed at all the good work you’re doing here. Nice improvements. I tested on both Mac and Windows. The “removethis.JPG” hack works great on both Mac and Windows.

On a Mac, using Image Capture:

On Windows, using File Manager:

probably need to rename the EXR file unfortunately and not make a copy.

Hee Ton,

Did you get the z1 in holland or did you order it from abroad? As soon as I get my hand on it i will try to support it.

Aan vr, mei 3, 2019 om 15:19, Ton Habraken via THETA 360 Developer <> schreef:

1 Like

Ha Kasper,

Ordered it at Cameranu.nl :slight_smile: … If you’d like to try with mine, let me know, I’ve not planned a shoot next couple of weeks…

Cheers, Ton

Hee Ton,

Ik dacht dat ze nog niet te krijgen waren in Nederland omdat er steeds staat dat ze tijdelijk uitverkocht zijn. Maar ze krijgen dus waarschijnlijk steeds maar een paar binnen. Dan ga ik hem gewoon bestellen.

Thanx voor het aanbod voor het lenen maar ben nu toch even druk dus wacht wel even tot we er een hebben op de academie.

Thanx!

Aan do, mei 16, 2019 om 10:04, Ton Habraken via THETA 360 Developer <> schreef:

HDR2EXR Plug-in Now available in the THETA Store

Everyone can download and install!

Developer mode not needed.

Description

HDR2EXR is a plug-in for RICOH THETA V to help visual special effects professionally easily grab lighting conditions on-set. Just by pressing the shutter button once, the HDR2EXR plug-in will take 11 bracketed pictures and merge them to one HDR file. The main use is intended for on-set VFX HDR capture.

Details

HDR2EXR measures on-set lighting conditions and bases bracketing on that specific lighting situation. It first takes 1 auto exposed picture to determine basic exposure settings. Based on that information, it sets the lowest ISO and then starts taking pictures.

For each bracket it takes 3 pictures and averages them together to reduce noise. It takes 11 brackets with a 2.5 stop increase. This should be enough to capture almost every lighting situation. Unfortunately, very bright light (like the sun) is still visible with lowest shutter times and ISO. This maybe can be fixed on the new RICOH THETA Z1 with higher aperture settings. This version of the HDR2EXR plug-in does not work with the Z1.

Finally, HDR2EXR automatically merges those 11 pictures into one EXR file ready to be used in NUKE, MAYA, etc. Creation of the EXR file is done by utilizing OpenCV HDR libraries.

Installation

And make sure to set the plugin as the default plugin to use.

How to use?

Start the plug-in by holding down the Mode button for 2 seconds. The little LED will turn white. And the Wi-Fi LED will turn Magenta.
Place the camera in the chosen location (use a tripod, shooting handheld will lead to crappy pictures) and push the shutter button. You have 5 seconds to run away and hide, else you are in the picture.
The Wi-Fi LED will turn green and the THETA will make picture taking sounds. It takes 34 pictures (1 to measure lighting and 3 times 11 brackets).
After the picture taking the Wi-Fi LED will blink red and blue. You can now move or pick up the camera. It is busy merging the pictures. This takes about 1-2 minutes. When it is done it will make a sound and the Wi-Fi LED will turn Magenta again.
Connect the THETA to a computer to download the pictures.

Good to Know

After running the plug-in, reboot your camera. The new files will not show up properly on Windows or Mac without rebooting.
In order to be copied and moved over to Windows, the EXR file is first created with a .JPG extension. It comes in this format: “EXRFILER05071408_removethis.JPG”. On Windows, using File Manager, copy over to your computer and then change the extension to .EXR. On a Mac, use the app Image Capture. Both the EXR file and the “EXRFILER05071408_removethis.JPG” will be available to copy over.
The default sleep setting for THETA is often 3 minutes. You should use the RICOH THETA mobile app to set sleep to OFF.
HDR2EXR tries to keep the ISO as low as possible but also the exposure time. When exposure gets above 1 sec, it increases the ISO until it runs out of ISO and then increases exposure time again. :wink:
This version works with OpenCV 3.4.4. I ran into to some problems with 4.0 which I couldn’t fix right away. It also generates a tonemapped jpg, just for fun. I haven’t been able to get this jpg to show up in the THETA iOS app. Don’t know why, maybe someone can help?
If you want to build it for yourself make sure to change the file paths in the Android.mk file (in the app folder).

Credits

The picture taking part is largely based on the work of Ichi Hirota’s dual-fisheye plug-in GitHub - theta360developers/original-dual-fisheye-plugin: Ichi Hirota's dual-fisheye plug-in for THETA V. Sample code for the THETA developer workshop. significant modifications by community
The integration of OpenCV is a combination of RICOH Blog Post: "Running OpenCV in Your RICOH THETA" and Setup OpenCV SDK in Android Studio project and a lot of trail and error!
The HDR part is based on https://www.learnopencv.com/high-dynamic-range-hdr-imaging-using-opencv-cpp-python/
Feel free to change, improve and of course use!
Let me know what you think and run into!

4 Likes

Yo,

The following is a post of unfortunately limited utility, but just to offer a round of applause for building such a useful tool.

I’m a 3D artist drawn to the theta for making speedy HDR maps on set for post production use. With the new Z1 hardware, what’s not to love?.. Well, the accompanying software. The promise (or perhaps I misunderstood) of easy HDR creation at the click of a button is tantalising, but given the current SW experience it’s a promise that isn’t delivered.

Awkward tooling, poor documentation (my favourite thus far has been a reference “See the help”… in the help), and a few errors that I’m currently unable to deal with, and I cannot at the time of writing take my exposure bracketed snaps and transform them into something useful. Hugely, hugely frustrating. Again, great hardware on paper, and the potential for a super-useful solution to HDR creation on a budget, but I just don’t feel as though the current SW experience has my back.

HOWEVER! The existence of this tool fills me with hope that the answer to my current nightmare may one day be realised.

Thanks again,

2 Likes

@FridayMarch26th, thank for the note of hope.

Please note that the plug-in does not currently work on the Z1. I think that only a minor modification is needed to the threading to the larger Z1 image sizes. Do you have a THETA V or a Z1?

If you have a THETA V and use this plug-in, make sure you understand that you need to change the extension from *.JPG to *.EXR on your local computer.

If you have any questions about the plug-in, please post them here.

Let’s keep the hope alive!

I’m a Z1 owner (…of around two hours), and yes, I note that the Z1 isn’t yet supported by this tool.

Really, I just wanted to note somewhere that the current Z1 software is not good, while sounding up beat about the future of the product.

So fingers crossed for that future. Let’s hope that as time passes development continues and the SW catches up with the HW.

3 Likes

@FridayMarch26th, thanks for the note. It did sound upbeat and hopeful. :slight_smile:

I can provide a little assistance with the following:

  • if you want to provide more specifics on the limitations of the Z1 software, I can pass the information to the product manager. In particular, are you referring to the workflow of using the mobile app to take bracketed shots (how many?) and then using Photoshop / Lightroom (or maybe specialized software) to create the HDR file. From there, do you put it into something like MAYA?
  • as Kasper was nice enough to open source the software, I was able to test it on the Z1. @jcasman and I may be able to help Kasper figure out where the problem is. I think the problem is minor. If Kasper fixes the problem, I can definitely help him to load an updated plug-in to the store

Additional note. I only looked at the problem for 10 minutes, but I think it may be related to the larger Z1 images taking up more processing cycles and I think it may just require the “merging” process to be run on a background thread (which is fairly simple to implement). I’m hopeful that it’s an easy fix. :-0

@FridayMarch26th, thanks for the feedback and all the details. Can you point out this self-referencing help link that you’re referring to? Is this in the RICOH THETA mobile app? The HDR2EXR documentation? Where did it come up?

Will it work with version 3.00.1 firmware?
I was able to shoot, but removethis.JPG is not generated.

The reverse of this technique could work for the EXR file.

Here’s a skeleton to run an HTTP server inside of the camera. It would need to be adapted for the specific file format of EXR.

Here’s an article and longer example of using a HTTP server that runs inside the camera.

There may be a problem with firmware 3.00.1. I’m checking it now. The plug-in is stopping with a pi-pi-pi error sound, which is not normal behavior. Asking @jcasman to help with testing now.

I will also test with firmware 2.50.1, which was the last version of firmware tested before submission.

test with firmware 2.50.1

Test 1

result: successful
firmware: 2.50.1
reboot: yes


Test 2 - no reboot

result: stuck on red flashing LED with no blue LED mixed in. No HDR file produced
firmware: 2.50.1
reboot: no

  • took out of plug-in mode
  • set back into plug-in mode
  • took three attempts to put into plug-in mode (came out of plug-in mode after a second on two attempts)

Test 3

result: success
reboot: yes
firmware: 2.50.1

WiFi LED solid magenta

  • 0:50 flashing green and taking pictures
  • 0:50 to flashing red with blue LED blinking occasionally
  • 2:00 blue flashing
  • 2:30 red flashing
  • 2:40 ending sound. One continuous sound. Solid Magenta
  • press lower mode button to take out of plug-in mode


Test 4

test condition: delete all old files from camera. reboot power of camera by holding down power button.
result: successful
firmware: 2.50.1
reboot: yes

Wi-Fi LED is solid magenta at start of test

  • 0- 0:50 - flashing green
  • 0:51- 2:05 flashing red with some blue
  • 2:06 - 2:40 flashing blue
  • 2:41 - 3:00 flashing red
  • 3:01 - end sound, solid magenta

Take out of plug-in mode by pressing lower mode button.
Connect to Windows 10 computer with USB cable.

Testing HDR file with Luminance HDR. File appears to contain all relevant light information.


3.00.1 firmware tests

Note: first three tests failed. Plug-in completes all shots per the procedure above. Does not save HDR file to camera storage. Will stop working with “pi-pi-pi” sound

Error using debugger

2019-05-31 04:26:14.488 2454-2454/com.kasper.hdr2exr I/MainActivity: Done with picture taking, let's start with the HDR merge.
2019-05-31 04:26:14.488 2454-2454/com.kasper.hdr2exr D/MainActivity: times length is: 11
2019-05-31 04:26:14.489 2454-2454/com.kasper.hdr2exr I/MainActivity: Merging average pics for denoise.
2019-05-31 04:26:14.614 2454-2454/? D/AndroidRuntime: Shutting down VM
2019-05-31 04:26:14.629 2454-2461/? I/art: Debugger is no longer active
2019-05-31 04:26:14.629 2454-2461/? I/art: Starting a blocking GC Instrumentation
2019-05-31 04:26:15.175 2454-5696/? W/art: Thread attaching while runtime is shutting down: Binder:2454_3
2019-05-31 04:26:15.175 2454-5696/? I/AndroidRuntime: NOTE: attach of thread 'Binder:2454_3' failed
1 Like

I’ve tested the HDR2EXR plug-in on a THETA V running firmware 2.50.1. This means it’s currently one update behind the 3.00.1 firmware.

The plug-in works well for me in multiple tests over several hours.

  1. Downloaded and installed HDR2EXR from the store using USB
  2. Set it as the boot plug-in using the RICOH THETA desktop app using USB
  3. Set THETA to Plug-in Mode - Main front LED goes from blue to white and the Wi-Fi LED turns magenta (pink)
  4. Shutter button starts the plug-in, it takes 34 images - This takes a little time - Approx 1 minute
  5. When it’s done taking the pictures, while is it busy creating the HDR file, the Wi-Fi LED blinks both blue and red - This also takes a little time - Approx 2 minutes
  6. When the HDR file is created, the Wi-Fi LED returns to magenta (pink) - I did this multiple times in a row - Sometimes I waited several minutes or sometimes I pressed the shutter button again almost immediately - Works fine
  7. Note: You need to reboot the THETA to see your HDR file - This is true for both Mac (using Image Capture or RICOH THETA File Transfer for Mac) and Windows (using File Manger)

Summary: HDR2EXR works for me using 2.50.1 firmware. If you want to use it, I believe it is not “easy to use,” but it does, with just one button press, nicely find the lighting settings and then use those to set the ISO for 11 bracketed images, and then produces an HDR file.

Getting that file off the THETA requires a reboot. And you need to understand that you then pull the file off the camera, rename the extension (format is this: “R05311154.EXR_removethis.JPG”) from .JPG to .HDR in order to get what you want. It’s not a big challenge, but it requires following the steps.

Example from Windows File Manager showing 8 HDR files (“R05311154.EXR_removethis.JPG”) inside the THETA ready to be moved to my computer

How can I delete on Mac folders created by HDR2EXR plugin ? ( the files I can delete with Photo App on Mac, but the folders are still on the Theta V.

Use the “Image Capture” app on the Mac. Plug the THETA into the Mac with a USB cable. Please post additional questions if you have any problems.

1 Like

@Zsolt_Horvath

Here’s my current understanding:

I cannot delete the folders on a Mac. Using Image Capture (or, as you suggest, the Photos app) I can delete all the pictures and .txt files. The folders will remain. Each folder takes up 4K.

I can see the folders by using the RICOH THETA File Transfer for Mac app, but I am not be able to delete them.

On a Windows machine, I can use the normal File Manager to see the folders and delete them.

1 Like

I’m not using a Mac. What do you mean, “I cannot delete the folders on a Mac?”

Are you suggesting that the file structure of the folder name can’t be deleted? Isn’t there some want to delete it from a Mac?

HDR2EXR plugin create folders and put photos into the folders. after downloading the photos I cannot delete the folders.