When people ask what’s most important when making a video, we start off by saying the word audio. Audio can decide whether or not your project is successful. Audio is what drives a scene and draws audiences in. In the last year, a large new development for online video came about, and that development was 360-degree spherical video. These videos are created with the use of a 360 camera rig, which consists of several cameras mounted together to cover all the angles around them. Once the footage is stitched together in post, it creates an interactive spherical video that allows viewers to look around using a smartphone or web browser.
Spatial audio
Simply put, spatial audio makes the viewer aware of what is happening and where in the scene it’s taking place. When edited correctly, spatial audio will give viewers an idea of where to look in the spherical video in order to see what’s going on in that particular area. Audio being positioned according to locations in a video will make a spherical video an even more immersive experience.
The first step in creating a spatial audio track for your spherical video is the actual recording of the audio. Many creators will use several different microphones when filming their spherical videos. If actors with dialogue are involved, they usually have lavalier microphones attached to them. This way the individual audio for each actor is caught without any interference from other parts of the scene. This type of audio recording works best for spherical videos because it allows the actors to move freely around the scene without being worried about whether or not they are being picked up by a mic.
The rest of the audio for the scene can be recorded by boom microphones placed underneath the 360 rig in different directions to capture other forms of production audio, or you can use a microphone that records in an Ambisonic format, which results in 4-channel audio capturing left/right and forward/back, unlike the standard polar pattern of a boom mic. A good step to take when recording audio like this is to label each lavalier microphone track according to which actor is wearing them and label the booms under the rig, as well. Along with this, make sure to give individual audio cues for each mic so that you know which is which during the editing process.
Post-production
There are several editing applications that allow you to edit audio for spherical video. A popular audio editing and mixing application is Reaper. It offers full multi track and MIDI recording, editing, processing, mixing and includes a mastering toolset which is perfect for editing together all of the audio tracks you will have for your video. One of the appealing features of Reaper is that it is free and works on both Windows and OS X machines. Being able to correctly layer and manage multitrack audio is essential to creating good spatial audio for a spherical video, but having the audio edited and exported isn’t all that needs to be done. You also have to place each part of the audio to match the locations in the 360 spherical world of your video.
This task can be taken care of with the use of a plugin for Reaper called Ambisonic Toolkit (ATK). The good thing about this plugin is that it works with Kolor Eyes, the free multi-platform 360 degree video player that lets you interact with 360 degree videos, zoom in and out, as well as look at what is taking place all around you. Using the plugin along with Kolor Eyes means that you can have a spherical representation of both your video and audio and rotate or move your audio on a 3-axis basis in order to simulate that full spherical audio track. Making sure that your audio and video are synchronized properly is also an easy task; a synchronization tool has been created by Spook FM that can be set to learn the rotation and zoom points of your video and apply them to the Reaper tracks in real time, giving you the option of positioning your audio to fit the spherical dimensions of your video.
There are plugins and software options available, as well, and new tools are still in development, so do some research to find the best tool for your purposes.
Publishing your work
Once you have your audio and your video aligned to match the spherical layout, it’s time to export them together and upload this to the platform of your choice. The process of mixing and merging the audio with the video is called muxing. You can do this with software like iFFMpeg. It allows you to export different video types and codecs for the various platforms that you might be publishing your video on. If you are uploading your video to YouTube, you can use a 360 Video Metadata App, which works for both Mac and Windows. This app was created specifically for YouTube and allows you to save your video file with new spherical metadata that will allow it to play correctly.
Recap
Hopefully, this article has given you an idea of how to go about creating spatial audio for your 360 degree videos. The important factors to remember when creating audio like this are the way in which you record your audio, how you edit your audio to match the spherical layout of your video and the encoding and exporting of the video so that it can be uploaded to the platform of your choice. Do it right and you’ll make your audience feel like they’re in the scenes you’ve created!
With a passion for filmmaking and exploration, Antonio is always trying to push the limits of the industry, often working as a one man film crew using DIY techniques where budget was not available.
