Small, wearable action cameras put things into perspective. Yet perspective tricks the brain unlike anything else. An object positioned with the correct perspective will appear firmly locked into reality. That same object can be a simple illusion, one that's created through perspective.
The video editor is more responsible now than ever before, for creating dynamic motion graphics. Applications such as Adobe After Effects provide all the tools necessary to generate amazing work. 3D camera tracking is a technique that helps the video editor put things into perspective and alter the reality of the audience. Camera tracking used to be a specialized task, one that only a select few could pull off. Now a video editor can perform 3D camera tracking on POV shots using the everyday tools that are at nearly anyone’s disposal.
Putting It in Perspective
People are used to seeing things within space, through perspective. Perspective relates to how objects are arranged and spaced in three dimensions. Distances are judged and the size of objects are estimated as a result of perspective. Perspective helps make sense of the world. Oddly enough, the human brain is able to extrapolate 3D space from two dimensional images, like video. One of the best ways to understand 3D space in a 2D medium is to visualize and track movement.
Perspective relates to how objects are arranged and spaced in three dimensions.
As visual perspectives change for an audience, they are able to better interpret and perceive 3D space from the flat image. 3D camera tracking works in much of the same way, it tracks points within the video image and measures how those points change over time. It then takes the data from the various tracked points and is able to solve their spatial relationship based on the 3D geometry of the points' positions. Once the 3D camera tracker solves the geometry of the scene, it is able to estimate the position of the camera that shot the scene and various planar surfaces that relate to the tracking points. Video editors and visual effects artists use this information to seamlessly composite effects into their video. The resultant imagery is convincing to the audience because the perspective is consistent throughout the footage and matches up to the camera's movement.
3D Camera Tracking
The technique of 3D camera tracking serves many purposes, all of which allow the video editor to create reality on the screen. It allows animators to insert 3D computer generated objects into a scene, extending sets and manipulating environments. Compositors are able to switch out signs and add reflections on objects. It gives the video editor the ability to add graphics and text that remain fluid with the movement of the videographer's camera. When motion graphics, text and design elements are tracked into first-person video, the graphics become part of the visual vernacular. It allows the audience to get swept up in the action and keeps them from being distracted by the editor's intrusion on the screen.
There are numerous plug-ins and tools on the market for camera tracking, some are for specialized compositing situations. Regardless of the tool used, the principles of the technique are the same. The software, whether it's a plug-in or stand-alone application, tracks points and/or regions that move through a clip of footage. The software extrapolates this data and uses it to solve for the geometric space of the content in the image. The solve allows the software to ascertain the location and movement of the camera that shot the footage.
Running Down the Tracks
Any good camera track is going to start with good footage. The software needs points and regions within the footage that it can identify and discern from other portions in the footage. Trying to track a snowball on a white cyc is difficult to do. In general, the footage should be well-lit, have some shadows and highlights, and for clearly outlined objects, a faster shutter speed should be utilized. Excessive motion blur, from either the camera or elements moving on screen, makes it more difficult to track.
Once the footage is ready for the software to perform the camera track, each frame needs to be analyzed. Some software, such as the 3D camera tracker effect in After Effects, will select its own tracking points as it analyzes the footage. Other camera trackers, such as mocha, will analyze the footage after the user selects planar surfaces, regions, and points for the software to track. This is why tracking points are used in chromakey videography for compositing on virtual sets; it helps to define the space for the 3D camera tracker.
After the footage is analyzed and the software solves for the position of the camera and geometric space of the scene, it generates data, rebuilding a corresponding scene in 3D space. This information is used by the compositing software to insert objects into the scene. In the case of a stand-alone application, like SynthEyes, it generates a dataset for export which is interpreted by another program, such as a 3D animation program like Maya. 3D camera trackers that are plug-ins or native to the compositing application, like the 3D camera tracker effect in Adobe After Effects, will generate data and a virtual camera that matches up with the scene. In the case of After Effects, the video editor can generate nulls and cameras, parenting text and other motion graphic elements to the nulls to build a matching 3D scene.
Keeping It in Perspective
One of the great aspects of 3D camera tracking is that it is independent of the final composition. A video editor can utilize the technique to insert motion graphics, flat 2D text, extruded 3D text, or objects of their choosing. This presents a wide array of possibilities. Examples of the results include the holographic projections of Tony Stark's lab in the Iron Man movies, the opening shots of Stranger Than Fiction, and just about every movie trailer which contains a reviewer’s appraisal in big bold metallic text spread throughout a cityscape. High energy action sequences from extreme sports go great with 3D camera tracking. ESPN Sport Science uses the technique to insert data driven infographics, making for a unique take on movement and sports.
As long as audiences desire to live vicariously through the videographer’s lens, there will be first-person point-of-view camera work. Action sports, even with its erratic movements, isn't necessarily sloppy, in fact, it can be quite exhilarating. A video editor doesn't need to avoid footage that's handheld where the camera becomes its own character. Video editors can embrace it and use 3D camera tracking to add graphics and effects that are part of the action, and enhance the story.
SIDE BAR
Tools for 3D Camera Tracking
Adobe After Effects CC
Adobe After Effects has had tracking capabilities for a long time. The native point tracker in After Effects is used by video editors to insert motion graphics and stabilize footage. However, it doesn’t create a true 3D track. After Effects now has an effect that is a true 3D camera tracker. The 3D camera tracker effect analyzes footage and is able to generate a virtual camera as well as insert null objects, solids, and text. It's a native tool that is helpful for compositing motion graphics.
mocha Pro
mocha AE has been bundled with Adobe After Effects for several generations. mocha Pro is a stand-alone planar tracker, offering much greater functionality than mocha AE. Both mocha AE and mocha Pro are popular options. It tracks user defined planes, surfaces and regions within a video image, rather than specific points. This provides a well defined and accurate track. The significantly less expensive mocha AE is well suited for motion graphics work, while mocha Pro adds better object removal and visual effects compositing.
The Foundry's CAMERATRACKER
CAMERATRACKER, from The Foundry, is a long established After Effects plug-in for camera tracking. It does have a few advantages over the After Effects 3D camera tracker effect. CAMERATRACKER gives the user greater control in refining their video and solves for, and corrects, lens distortion. It is purchased separately from After Effects and used in the industry for some of the most demanding projects.
SynthEyes
SynthEyes is a stand-alone 3D camera tracking application that is available on the Mac, Windows and Linux platforms. It's an incredibly powerful program that is popular with 3D artists. It's able to output data that can be used in the most popular compositing and 3D animation programs. SynthEyes users utilize it to help composite 3D animation with live action video footage, reconstruct or replace sets, build 3D textured meshes, and place 3D generated product shots. It is a high-end piece of software from Andersson Technologies for specialized compositing work.
Contributing editor Chris “Ace” Gates is a four time Emmy Award-winning writer and producer.
