The Little Black Box

Bob_White · June 4, 2017, 7:55pm

I have never been happy with just standing my Theta S on a table or shelve as it’s shape while very functional for shooting pictures just does not lend itself as being stable when placed on a flat surface. I tried using inexpensive table tripods and they were an improvement but still not quite what I was looking for.

So the birth of the “Little Black Box” (LBB).

I started with a project box which measures 4.4” x 2.4” x 1.1” and some hardware including a 1/4 20 thread metal rod so that I could incorporate the designs offered elsewhere on this forum to allow using the camera’s mount thread while still having access to the camera’s power and hdmi jacks.

Here is the finished product depicted with it’s short mount and with it’s short mount with added extension.

But wait, there is more!

The LBB needed to have some ballast to increase it’s stability when configured with it’s extension rod so I packed the box with some goodies.

The main components are a 2500mA LiPo battery, an Adafruit PowerBoost 1000C, a Raspberry Pi Zero W, a Professional Ultra SanDisk 64GB MicroSDXC card and a Edimax EW-7811Un 150Mbps 11n Wi-Fi USB Adapter.

Now the LLB has a lot more heft then the Theta S.

Well, that is it for the major pieces of hardware for this project. Anyone care to guess which piece of hardware was the most expensive?

What about the software?

I started with the Python code that I wrote for the Time-Lapse challenge and am still expanding from there. I plan to write up the software side after it becomes more mature but it is currently functional for a limited group of functions.

The Pi Zero W is setup with the following packages:

Raspbian Jessie
Apache 2.4.10 webserver
PHP 5.6.30
Python 3.4.2
Samba 4.2.14
SSH
Google VRview

The Pi Zero also has two WIFI networks. Wlan0 is used to talk to the Theta S or when the Theta S is not connected to my local network. It is set so that the Theta S has priority if both networks are available for connection. Wlan1 is an access point used to control the Pi via web browser, SSH, or to download pictures from the Pi with Samba.

The following pictures will give you some idea where I am at with the software/web development side of my project.

Notice the ability to view pictures as either flat jpegs or as 360 VR. I can even view them in Google Cardboard on my iPhone direct from the LBB index of Pi ThetaS-Pics.

Lastly here is a link to a Theta360.com photo from my last testing. The photo shows me viewing photos while the camera is still capturing photos from the 10 photo command that I had asked for. I am viewing photos on both my iPhone and iPad in 360 VR within seconds of them being shot and while the camera is working on its 10 picture run. This feature would be really nice to check status or just grab pictures while on a really long run.

In this run I am using internal power for the Theta S and for the LBB PI but the LBB is capable of having external power (5V micro USB) applied to it as well. When the LBB is receiving external power it can also power Theta S from a USB connector on the side of the box. The LBB charging and boost circuit and associated LiPo battery will ensure that the 5v applied to the Theta S is a clean and constant 5v.

Post from RICOH THETA. - Spherical Image - RICOH THETA

jcasman · June 5, 2017, 4:40am

Wow, this is cool! Thanks for posting all the hardware details. Your case is really clean and compact, I’m impressed. Thanks for posting the exact weights, too. That’s kind of interesting to take into account to make sure it’s super stable but not unwieldy.

The Pi Zero also has two WIFI networks. Wlan0 is used to talk to the Theta S or when the Theta S is not connected to my local network. It is set so that the Theta S has priority if both networks are available for connection. Wlan1 is an access point used to control the Pi via web browser, SSH, or to download pictures from the Pi with Samba.

I think this is really key to your Little Black Box (LBB) and probably key to many hardware setups that use the RICOH THETA. I actually don’t know how to set it so that the THETA has priority. Is that easy to do?

Bob_White · June 5, 2017, 5:33am

Just add a line for each network that you have listed in /etc/wpa_supplicant/wpa_supplicant.conf with a priority assigned.
The higher the number the higher the priority.

The LBB/etc/wpa_supplicant/wpa_supplicant.conf file looks like this:

country=US
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
   ssid="Whitehouse"
   psk="My networks password"
   proto=RSN
   key_mgmt=WPA-PSK
   pairwise=CCMP
   auth_alg=OPEN
   priority=1
   id_str="home"
}

network={
   ssid="THETAXS00xxxxxx.OSC"
   psk="Theta wifi password"
   proto=RSN
   key_mgmt=WPA-PSK
   pairwise=CCMP
   auth_alg=OPEN
   priority=2
   id_str="ThetaS"
}

Svendus · June 5, 2017, 7:39am

Nice box Bob well done a small circular mirror with a center hole just under the THETA
can hide the little mystical box
@Bob_White got the idea when i saw your rig
we shall look after a smaller mirror
wonder the diameter mirror

When are the box ready for production ?

codetricity · June 5, 2017, 3:46pm

Nice job. Would be great to get your source code repository whenever you’re comfortable with sharing it. This is something a lot of people would be interested in. Are you using ssh to push the images to a public server as well for viewing by the public?

This is cool.

BTW, nice shirt.

Bob_White · June 5, 2017, 4:46pm

Thanks. Any thought of production is a ways off as I still have a lot of code to write on the LBB to Theta S control side. I am hoping to implement the full Theta S API through Python scripts. I would like to have the capability to easily configure the Theta S through the use of templates via a webpage.

Are you ready to order?

Bob

Bob_White · June 5, 2017, 4:55pm

Craig,

Yes, it is a nice shirt. I really appreciate my shirt and only wear it for special occasions.

The source code is a long way from being completed but I will put together a block diagram about how all the pieces fit together after I watch the Apple WWDC keynote this morning. A quick answer is that the images are first stored on the Pi Zero in a public folder and viewed through it’s Apache web server. Images can then later be pulled via Samba or pushed via a python script and sftp to a long term storage server.

Bob

Bob_White · June 5, 2017, 8:23pm

Little Black Box Structure

Bob_White · June 8, 2017, 10:22pm

I was taking a little breaking from coding on the little black box today and was playing with Affinity Photo which is my latest iOS app purchase. I have the app installed on an iPad Pro 12.9 generation one. I wanted to see how easy it was to change the starting center of image on a 360 image captured with the Theta S and it was also a good time to test the Little Black Box Samba function. So I got on my MacBook Pro and connected the finder to the Little Black Box via my home network and was greeted with this screen.

Selecting the ThetaS-Pics subdirectory produced this:

I then selected a few photos (17 to be exact) and pasted them to a Theta-S subdirectory I have set up on iCloud.

After the transfer I pulled up the Affinity Photo app on my iPad Pro and added one of my iCloud images using the Affinity Photo Import menu.

Here is the center of picture for the original image:

Moving the center of image and removing the Little Black Box from the photo was a snap.

First select Equirectangular Projection from the Projections menu.

Then find your new center of picture and select Centre Coords.

That’s it for setting the new center. Now for removal of the Little Black Box footprint.

Select Inpainting Brush from the brush menu.

Paint out the area to hide.

And you are done!

Another nice feature of Affinity Photo app is the ability to display image metadata.

The Little Black Box moves pictures out of the Theta S camera storage area as soon as the pictures have finished being processed. The images are then immediately available from the LBB storage area for download. I envision that during a long Time-Lapse runs I will be checking images from time to time and with the easy availability of image metadata, I will be making realtime decisions on what camera parameters need to be tweaked during the run.

Now I have something else to code. Implementing realtime camera changes during the course of a run.

I want to send out a special thanks to @Svendus for pointing out the available of the Affinity Photo app for the iPad while it was still on introductory sale. I think it still might be on sale for anyone else that is interested. It does require an iPad Air 2 or newer. I know for a fact that it will not run on the original iPad Air as I tried it even though the product clearly states that an iPad Air 2 or newer is required to run the app. I just had to try.

jcasman · June 9, 2017, 11:27pm

Bob,

Reading through your description and comments, I have a couple questions. They kind of jump around, but I’m curious if you have any feedback.

Referring the quote above, what types of camera parameters do you think you would be tweaking in real-time? You’re talking about adjusting ISO settings, shutter speed, etc.? This would mostly be for adjusting to darker light conditions?
I’m a little surprised that the “balast” for the base is actually components. I guess you have no need for adding more weight? Like a chunk of lead or a roll of coins or something?
In your time-lapse submission, you strapped your camera rig onto a fence with bungie cords. Do you envision a square box as a better type of base? (I guess it totally depends on use-case scenarios. Where do you think you’ll be installing the LBB?)
I’m really enamoring with the two network solution. In fact, I think it’s essential. It’s really as easy as adding a USB wifi dongle and editing network priority settings? Was there anything else involved in setting that up?

In this run I am using internal power for the Theta S and for the LBB PI but the LBB is capable of having external power (5V micro USB) applied to it as well. When the LBB is receiving external power it can also power Theta S from a USB connector on the side of the box. The LBB charging and boost circuit and associated LiPo battery will ensure that the 5v applied to the Theta S is a clean and constant 5v.

I just found out power over USB and date over USB are different things. This is new to me. I guess it’s not much more of an issue than making sure you have the right cable. For a different device (Renesas S3A7 Fast IoT Prototyping Kit) I’m using an external portable battery (2200 mAh capacity, 5V / 800 mA DC input and output). Cool for temporary solutions:

Thanks for any help,
Jesse

Bob_White · June 10, 2017, 12:29am

Jesse,

Exactly. Modify camera settings on the run to match conditions. And since the LBB let’s you look at the output anytime during a run you can have a good idea what you need to modify. I have also just added the ability to run exiftools on the pictures residing on the Pi. Here is an example (parts are cut off but the text area is a scrollable box).

The weight as is seems fine for the stand and extension that I am currently using.

When I started doing the time-lapse work I was just controlling the camera with my iPhone and running pythton scripts on the phone to control the camera and store the images. This works but it ties up the phone and limits the distance between the camera and phone due to low wifi power and poor antennas. I then moved on to adding a Pi zero W to control the camera and then a second wifi circuit to talk to the Pi zero W via SSH so that I could modify scripts on fly and move pictures off the pi at will. Here is a pictures of that setup. Just add power, selfie stick and bungee cords.

Between the two systems I have lots a flexibility now. Of course all the upgrades that I have done system/software wise can be applied to both hardware configurations.

Minimum you need to set up the Pi to auto connect to the Theta S on one of the wireless networks and you also need to have a wireless access point set up on the second wireless network. The access point is where you can log in (with multiple devices) to do things on the Pi. At a minimum you need to have SSH turned on so that you can use the Pi in a headless mode. With the LBB I have taken everything to the next step by adding a web server, PHP, file manager (Samba), Google VRserver, WIFI Command, and now ExifTools. Here is an updated map of what the LBB can do.

As for power, I believe the internal battery of the Theta S and the LBB can power the system for about seven hours. For really long runs I just add an external 5V power source. A bigger battery or a power adapter. One thing nice about the LBB power system is that it can take a “dirty” input and clean it up to keep the Theta S and Pi happy. The Adafruit charging/boost circuit and the LiPo battery built into the LBB do a really nice job of ensuring a nice clean and constant 5.11V output with no spikes.

I think that I answered all your questions. If not, fire away.

Bob

jcasman · June 10, 2017, 5:55pm

Getting all the metadata info on the Last Picture image should be really useful. I think people really care about initial orientation, and probably some other details. Do you have any problems with horizon correction? I’m guessing not, cause this is “all THETA.” But pointing out this post on horizon correction just in case.

It’s cool seeing the “MVP” package as well.

Also, thanks very much for the LBB components map. Very useful for understanding what you’re doing with your setup.

Also-also, I’m hoping to use your time lapse setup for doing some work in 360 astronomy.

Bob_White · June 10, 2017, 8:41pm

I took a quick glance at the 360 astronomy link. I will have to revisit it when I have a bit more free time. Thanks for the link.

I will eventually give the LBB menu system the ability to get full metadata on any image stored on the Pi’s memory card. It’s on the to-do list.

Let me know if you need any help with the time lapse setup while using my code. There were a lot of parameters that were hardcoded in that Python code.

The LBB will have all the camera option settings available from HTML pages soon. I am currently working on the retrieval of all API options from the Theta S for display via the LBB. The next step will be the providing the ability to change any of the parameters on the camera from the LBB via HTML pages. The last phase of my settings work will be providing the ability of building and storing Theta S setup templates for easy retrieval from pi memory and for application back to the Theta S.

I envision the templates as being a system to take a snapshot of how the camera is setup at a particular time so that you can apply the same settings as a baseline for another shoot.

jcasman · June 13, 2017, 11:08pm

First Offsite Test of the Little Black Box

Thanks to @Bob_White for allowing me to test the Little Black Box (LBB), “across state lines,” as he says. As a tester, to get started it was easy; I pointed my browser at the webserver housed in the LBB. Bob had temporarily opened up the firewall to all web connections.

After 10-15 seconds negotiating the initial connection, up came the LBB interface, with multiple pulldown menus and buttons.

In general, the responsiveness was very good. I noticed latency just once with the first connection, and then if you took multiple pictures, there is 5 seconds between pictures (the time it takes to stitch two hemispheres together on the camera) plus several seconds before the last pictures shows up in the LBB interface.

The pulldown menu for actions:

At first I tried one picture at high res. I can then choose that picture (or others already in memory on the camera) to see the full metadata. You can view a thumbnail (actually it’s the full image just displayed small) of the equirectangular version. AND Bob has implemented GoogleVR so you can easily see the full navigate-able 360 degree photosphere in your browser, too. Very cool!

The Index page shows the full list of images, in reverse chronological order. Note the bottom 7 are taken today (June 13, 2017) and include 1 high res image (highlighted in red rectangle) and 6 low res tests. Equirectangular versions are on the left, 360 photospheres on the right.

Through the LBB interface, you can run multiple API commands. Here I am running /osc/infoand getting back the current state of the camera:

Finishing up just required turning off the wifi which kills the outside connection.

Comment on power: With smaller electronic gadgets and IoT devices, it’s maybe easy to overlook power. The LBB can take power (cord or battery) to the Raspberry Pi Zero W and power the THETA. Only one power source needed. “It’s super clean,” says Bob.

Conclusion

If you are interested in setting up remote time lapse solutions with a RICOH THETA, you should strongly consider contacting Bob’s consulting firm. He has LBB feature-rich example already built. And his consulting business is open to taking on new challenges and adding different functionality.

Bob_White · June 14, 2017, 6:37am

@jcasman Thanks for all the help today testing the use of the LIttle Black Box via the internet.

Here is a new map of LBB services adding the remote via internet option.

jcasman · June 15, 2017, 4:59am

Bob, thanks a ton. Just posting quickly info about the app you used to map out visually all the services of the LBB. It’s called “iThoughts” by a small UK company called Toketa.

Toketa is a term used in Judo to describe when a person has escaped from being pinned down i.e. they’ve broken free. Seemed apt somehow - software that sets you free. That’s the goal anyhow!

Cool stuff.

codetricity · June 20, 2017, 8:13pm

When you use the Ricoh Desktop application for horizon correction, doesn’t it just zero out the meta data for camera orientation?

Prior to correction (original image)

After correction with Ricoh Desktop application

I think it zeroes out the image orientation data. I don’t think it corrects the actual image.

In the screenshot below, the top image is the corrected image and the bottom is the original.

The zero out is useful is you edit the image in an image editing application.

Bob_White · June 23, 2017, 8:40pm

The LBB has a second webpage under development.
Right now it will read the Theta S and set pickers to the current settings in the camera.
The next step will be to read changes made to the pickers on this webpage and to write these changes back to the Theta S.

General Information Only Options and Special Options can be viewed from the Tools Menu of the Control Panel. This data is currently displayed as raw data from the Camera. I will be cleaning up this output and also moving some of these options over to the settings menu. These options required slightly different code to extract them from the Theta S hence my orginally grouping them seperate from the main group of options.

codetricity · June 23, 2017, 8:58pm

Have you considered trying to sell or lease the system to a used car chain? This company uses the THETA.

You’ll need a way to create the virtual tour.

Another market is real estate.

In my opinion, the key is to start off with a vertical industry like auto sales, real estate sales, or surveillance and then make the process of taking images and then displaying them as easy as possible.

codetricity · June 23, 2017, 9:11pm

Another thing to consider is the upcoming THETA Developer Contest run by the RICOH HQ team. It’s a global contest. The winners of these contests often start their own business.

This was last year’s winners:

These are the winners from 2015

http://contest.theta360.com/2015/result/index-en.html

You can see that HoloBuilder converted this into a business.

https://landing.holobuilder.com/construction

The prestige and validation of winning the prize is probably more important than the financial award. Though, the financial award could be used to kick-start the business.