One of my favorite features of the Ricoh Theta V is its spatial audio. It’s the best that I’ve heard among my cameras, including Insta360 Pro. In the first part of this post, I will show clips to demonstrate spatial audio. In the second part, I will discuss how to edit spatial audio for post production.
WHAT IS SPATIAL AUDIO?
Spatial audio, sometimes called ambisonics, is a way to record a 360 audio sphere. With conventional audio, even 5.1 surround sound audio, the sound you hear does not change regardless of which direction you are facing. With spatial audio, you can hear the sound change as you look around. It’s perfect for 360 videos, because the direction of sound will match the direction you are viewing.
The best way to understand it is to experience it. Here is a 360 video demo of spatial audio (shot on the Theta V), and compared with conventional audio from a Theta S. You should use headphones or earphones to hear the effect as you look around the 360 video:
POST-PROCESSING WORKFLOW FOR SPATIAL AUDIO
The Theta V has built-in spatial audio, using four microphones. You don’t need to buy special equipment to record or listen to spatial audio but Theta V files need to be processed in a particular way to extract the spatial audio.
The simplest way to enjoy spatial audio is to play Theta V videos through the Ricoh Theta desktop app. There you will hear the spatial audio without the need to process the files.
To hear spatial audio in YouTube or Facebook, the Theta V files must first be converted using Ricoh’s Movie Converter app. Just drag the video file into the Movie Converter app icon and the app will create a copy of the video with 4-channel audio with a *.mov extension. If you upload this file to YouTube, it will have spatial audio. It should work for Facebook as well, although I haven’t tried.
If you want to edit your 360 videos, you will need a video editor that supports 360 spatial audio. One such editor is Adobe Premiere Pro, which added ambisonic support in CC 2017. Here is the workflow for Premiere.
First, you must convert your Theta V videos into the 4-channel *.mov files using the Movie Converter app.
Second, you must prepare your Premiere project to accept an ambisonic Theta V file. Go to Edit… Preferences… and under Multichannel Mono, change the option to “Adaptive.” Then create a new sequence (File… New… Sequence) and look under the VR preset for the monoscopic 29.97 ambisonic preset. You will see in the sequence settings that this sequence has a multichannel format with 4 audio channels.
Third, you need to import your Theta V files. When I did this the first time, I got confused because I only saw one audio track. But actually, if you expand the height of the track, you’ll see there are four channels within one track. At this point, you should orient each video file to its final orientation using the VR Rotate Sphere or other similar effects.
Fourth, you need to align the direction of the sound with the video because unfortunately, Premiere doesn’t automatically align the sound correctly with the sound of the video. On my Theta V, it appears that there is a 90-degree difference. You can hear the difference in the direction using the Binauralizer Ambisonics audio effect, which you can apply to the Master channel on the Audio Mixer. When the Binauralizer effect is applied, you can use the pan parameter to simulate looking around the video. (But strangely, as of January 2018, rotating the video will not change the sound, even in VR view.)
Once you have determined the difference between the direction of the video and the direction of the audiosphere, you need to rotate the audiosphere. To rotate the audio sphere, you need to use an effect called Panner Ambisonics. You simply drag the effect onto the audio clip (the 4-channel track). With the Panner Ambisonics effect, you can rotate the audiosphere’s pitch, yaw, or roll.
After confirming that the audio sphere matches up with the video sphere, you need to remove the Binauralizer Ambisonics effect (or else the audio won’t render correctly). You can then proceed with editing the video as usual.
Finally, When you’re ready to export, go to File... Export... and select the preset for VR monoscopic Ambisonics. You’ll see that in the video tab, the checkbox for VR monoscopic will already be checked. In the audio tab, you should see audio in AAC format with 4.0 channels, and a 512 bitrate. The check box for Ambisonics should also be already checked for you. When you click on Export, the resulting video will have spatial audio compatible with both YouTube and Facebook.