Stitching is a fundamental step in immersive video production. Stitching is a computational process that transforms the multiple fisheye videos captured from a VR camera into a spherical video suitable for playback in a VR headset. Each fisheye video is de-warped and arranged, before being blended into a single video in “equirectangular” projection, the standard projection used to encode a spherical image into a rectangle (it’s commonly used in maps).
3D-180 media is technically not “stitched”, but the term is commonly used as shorthand for the general process of converting captured media to a VR-ready format.
Specialized software tools or plug-ins are used for stitching. The process is very computationally intensive, so powerful hardware and graphics cards are beneficial. Stitching tools include features like color matching between lenses, optical flow, editable camera rotations and scaling, edge points (to force a stitch to use one lens instead of another), and image stabilization. Stitching tools sometimes require calibration data from the camera which captured the content; if this exists, it is usually encoded into the header of files written to a camera’s media card(s). These data may include what are called camera “intrinsics” and “extrinsics”: sensor and lens specifications, lens warp profiles, positional offsets of the lenses, etc. The metadata are used to assist in the stitching process.
Many camera manufacturers develop and distribute their own stitching software. These tools are often free for camera owners, and are meant to automatically stitch video from their cameras. However, stitchers from manufacturers are usually limited in features and lack the ability to customize stitches.
Third party stitching software supports customization and manual control of stitching. These tools require technical expertise for best results in use, but are commonly used by most immersive video productions. Examples include SGO’s Mistika VR, and in the high end, Foundry’s Nuke with CaraVR.
Regardless of how a creator plans to stitch immersive videos, it is a good idea to capture test footage to run through a planned post production workflow before committing to a camera and to software tools, both to stress test the production workflow as well as evaluate the camera and output quality.