Stabilizing 360 video in Mistika VR

This article provides an overview of stabilization in Mistika VR applied to an Insta360 Pro2 camera. The basics are applicable to any 360 video in Mistika VR.

If a VR camera is mounted to a rover, cable system or drone for a moving dolly shot, or experiences any instability during shooting, stabilization of the original footage is crucial for delivering a comfortable viewing experience in VR. When shooting with a moving camera, the use of specialized immersive media camera stabilizers can help reduce unwanted tilt and sway, but stabilizing in post will likely be required.

Mistika VR provides two methods for stabilization which are both straightforward to implement. Mistika VR can calculate stabilization by analyzing the footage’s horizon and vertical lines. The second method imports gyro metadata from Insta360 Pro 2 and Titan, KanDao Obsidian, and Teche360 cameras.

In Mistika VR, both methods of stabilization run on the Output Camera, generating a set of S-Yaw, S-Pitch and S-Roll values. These stabilization values will be displayed as keyframed values in the output camera. The Yaw, Pitch and Roll parameters act independently of S parameters and can be edited after stabilization for reorienting the footage or horizon leveling.

For more information on modifying the output camera parameters read these articles Leveling a 360 video in Mistika VR and Reorienting a 360 video in Mistika VR.

In Mistika VR, stabilization is a single click, with minimal options required. The stabilization process analyzes movement on the video’s horizon plus 45 degrees above and 45 degrees below. The nadir and zenith of the video are not tracked. If the scene is strongly tilted in the starting frame, it should be reoriented to a roughly horizontal position before starting stabilization. After stabilization, the Output Camera can be further reoriented, these parameter edits will be applied on top of the stabilization.

• Follow Overall Heading: when activated, stabilization will try to compensate for camera shake only, and will respect the overall heading of the rig. The camera will generally stay centered which permits stabilizing the camera along a curved path. If deactivated, the stabilization will try to keep the view as steady as possible, but the process may be adversely impacted by one side of the scene advancing while the other does not. In this case, the scene may gradually roll or tilt.
• Follow Heading After x Frames controls the number of frames the stabilization uses to alter the camera’s heading. Low values change heading faster, but may produce less smooth results. Higher values change the heading slower. If stabilization seems to drift too far from the intended orientation, reduce the value.
• Reapply Stabilize will reprocess the stabilization, based on the previous results, which can improve the results without needing a full rescan.

Set the In/Out frame marks to limit the stabilization time range to just the desired portion of the shot. Approximately level the horizon on a frame early in the range to stabilize, this will provide the stabilization process a reasonably leveled horizon to start with. Run the Stabilize command, and then level the horizon using the Yaw, Pitch and Roll parameters, which will be applied on top of the stabilization S parameters. Prior to the Stabilization scan, deactivate Optical Flow. Stabilization normally should be run after aligning the stitch and matching colors. Read this article to learn more about stitching and matching color in time.

Mistika VR supports import of the Insta360 Pro 2 or Titan gyro metadata which was captured during each individual shot. The Pro 2’s recorded metadata is stored in the pro.prj file for each shot and is utilized when selecting Import Stabilization Metadata under the Stabilize pop-up menu. To learn more about supported camera metadata, refer to the SGO Mistika VR website.

Import the stabilization metadata by clicking on Stabilize > Import Stabilize Metadata. After clicking on the button, the stabilization is imported and the footage is stabilized without time-consuming scanning and processing. As with processed stabilization, imported stabilization modifies the S-Yaw, S-Pitch and S-Roll parameters.

Recorded camera gyro metadata can sometimes be out of sync with the clip loaded in Mistika VR.

To fix this misalignment in time, SGO recommends following this procedure:

• Find a short segment of 1 or 2 seconds where there is a regular rocking movement with a consistent forward camera movement.
  
• Set In/Out points on the segment and click the play button. The segment will now play in a loop. Do not activate the optical flow.
  
• Find a part of the shot with forward movement, which exhibits camera wobble. Zoom in and center the frame using the right mouse and drag. Let the video continue to loop.
  
• With video playing in a loop, change the Import Stabilization Offset parameter to +5ms, and apply the Import Stabilization Metadata again. The play loop should now continue, and you can see if the stabilization has improved.
  
• If +5ms makes the stabilization clearly worse, apply -5ms value instead. Experiment with different options until reaching a value that works best for the shot.