How to Create Stereoscopic 360 Video with FFmpeg

This article provides a practical guide on using FFmpeg to assemble and encode stereoscopic 360-degree videos for Virtual Reality (VR) headsets. You will learn how to combine left-eye and right-eye equirectangular video streams into standard VR layouts, apply optimal encoding settings for VR hardware performance, and ensure the final file contains the appropriate metadata for proper spherical 3D playback.

Understanding Stereoscopic 360 Layouts

Stereoscopic 360 video requires two synchronized equirectangular video streams—one for the left eye and one for the right eye. These streams are combined into a single frame using one of two standard formats:

  • Top-Bottom (Over-Under): The left eye is placed on top and the right eye on the bottom. This is the industry standard for 360-degree stereoscopic video because it preserves full horizontal resolution per eye.
  • Side-by-Side (SBS): The left eye is on the left and the right eye is on the right.

Step 1: Combining Left and Right Eye Feeds

To combine separate left and right camera sources into a Top-Bottom layout, use FFmpeg's vstack filter.

ffmpeg -i left_eye.mp4 -i right_eye.mp4 -filter_complex "[0:v][1:v]vstack=inputs=2[v]" -map "[v]" -map 0:a? -c:v libx264 -crf 18 -pix_fmt yuv420p stereoscopic_360.mp4

To create a Side-by-Side layout instead, replace vstack with hstack:

ffmpeg -i left_eye.mp4 -i right_eye.mp4 -filter_complex "[0:v][1:v]hstack=inputs=2[v]" -map "[v]" -map 0:a? -c:v libx264 -crf 18 -pix_fmt yuv420p stereoscopic_sbs.mp4

Step 2: Scaling for VR Headset Compatibility

VR headsets decode video using dedicated hardware chips that have specific resolution and aspect ratio constraints. For Top-Bottom 360 video, a square aspect ratio such as 4096x4096 or 3840x3840 is standard (each eye gets a 2:1 aspect ratio of 4096x2048 or 3840x1920).

Use the scale filter to ensure your combined video meets these dimensions:

ffmpeg -i left_eye.mp4 -i right_eye.mp4 -filter_complex \
"[0:v]scale=4096:2048[left]; \
 [1:v]scale=4096:2048[right]; \
 [left][right]vstack[v]" \
-map "[v]" -map 0:a? \
-c:v libx265 -crf 20 -preset medium -pix_fmt yuv420p \
stereoscopic_4k.mp4

Step 3: Encoding Best Practices for VR

  • Codec: Use H.265 (HEVC) via libx265 for resolutions at or above 4K to maintain high visual fidelity without exceeding bitrate limits. Use H.264 (libx264) for maximum compatibility with older standalone headsets.
  • Pixel Format: Always set -pix_fmt yuv420p. Headset decoders typically fail to play 4:2:2 or 4:4:4 formats.
  • Audio: Keep the original audio using -map 0:a? or encode spatial audio (Ambisonics) to AAC or uncompressed PCM depending on the target player.

Step 4: Injecting VR Metadata

VR video players (such as those on Meta Quest or YouTube VR) rely on embedded metadata tags to identify that a file is spherical and stereoscopic.

FFmpeg allows you to set the stereoscopic mode flag directly into the MP4 container:

ffmpeg -i input_combined.mp4 -c copy -movflags +faststart -metadata:s:v stereo_mode=top_bottom output_vr.mp4

For Side-by-Side, change stereo_mode=top_bottom to stereo_mode=left_right.

If your target playback software does not automatically recognize container-level flags, inject standard Google Spatial Media metadata into the resulting MP4 using the open-source Spatial Media Metadata Injector tool, selecting both the "Spherical" and "3D Top-bottom" options.