Create Stereoscopic 360 Video with FFmpeg
This article provides a practical guide on how to use the command-line tool FFmpeg to merge separate left-eye and right-eye video feeds into a single stereoscopic 360-degree video suitable for VR headsets. You will learn the exact FFmpeg commands needed to stack your video files, configure the correct format for VR players, and prepare the output file for proper 3D playback.
Understanding the Stereoscopic 360 VR Format
To play a 3D 360-degree video in a VR headset, the left-eye and right-eye views must be combined into a single video file. The industry standard format for this is Top-Bottom (Over-Under), where the top half of the frame contains the left-eye projection and the bottom half contains the right-eye projection.
For 360-degree video, each eye’s input is typically in an equirectangular projection with a 2:1 aspect ratio (such as 4096x2048). When stacked vertically, the final video container will have a 1:1 square aspect ratio (such as 4096x4096).
Stacking Left and Right Videos Vertically (Top-Bottom)
To combine your separate left and right video files into a Top-Bottom
stereoscopic layout, use FFmpeg’s vstack filter.
Run the following command in your terminal:
ffmpeg -i left_eye.mp4 -i right_eye.mp4 -filter_complex "[0:v][1:v]vstack=inputs=2[v]" -map "[v]" -map 0:a? -c:v libx264 -crf 18 -pix_fmt yuv420p output_tb_360.mp4Command Breakdown:
-i left_eye.mp4 -i right_eye.mp4: Specifies the two input video files (left eye first, right eye second).-filter_complex "[0:v][1:v]vstack=inputs=2[v]": Takes the video streams of both inputs and stacks them vertically. The left eye[0:v]becomes the top half, and the right eye[1:v]becomes the bottom half.-map "[v]": Maps the stacked video filter output to the final file.-map 0:a?: Copies the audio from the first input file (the question mark ensures the command doesn’t fail if the input has no audio).-c:v libx264: Encodes the video using the H.264 codec, which is widely supported by VR headsets like the Meta Quest.-crf 18: Sets the Constant Rate Factor for high-quality output (lower numbers mean higher quality; 18–22 is ideal for VR).-pix_fmt yuv420p: Sets the pixel format to YUV 4:2:0, which is required for hardware decoding compatibility on most VR headsets.
Stacking Left and Right Videos Horizontally (Side-by-Side)
While Top-Bottom is preferred for 360 VR, some players require a
Side-by-Side (SBS) format. To stack the videos
horizontally, use the hstack filter:
ffmpeg -i left_eye.mp4 -i right_eye.mp4 -filter_complex "[0:v][1:v]hstack=inputs=2[v]" -map "[v]" -map 0:a? -c:v libx264 -crf 18 -pix_fmt yuv420p output_sbs_360.mp4Injecting VR Spatial Metadata
Simply stacking the videos is not enough for most VR players to automatically recognize the file as a 3D 360-degree video. Without spatial metadata, the player will display the video as a flat, split-screen video.
Option 1: Injecting metadata during encoding with FFmpeg
Modern versions of FFmpeg allow you to inject spherical metadata
natively using the -sv3d option. Run this command to encode
and inject the metadata simultaneously for a Top-Bottom (Over-Under)
setup:
ffmpeg -i left_eye.mp4 -i right_eye.mp4 -filter_complex "[0:v][1:v]vstack=inputs=2[v]" -map "[v]" -map 0:a? -c:v libx264 -crf 18 -pix_fmt yuv420p -sv3d "type=equirectangular:layout=top-bottom" output_vr_ready.mp4Option 2: Post-processing with Google Spatial Media Metadata Tool
If your media player does not recognize the native FFmpeg metadata, you can use the command-line version of the Google Spatial Media Metadata Tool to inject the metadata after rendering:
python spatialmedia -i --stereo=top-bottom output_tb_360.mp4 output_vr_final.mp4Once metadata is injected, transfer the final file to your VR headset or upload it to a compatible VR video platform for immersive 3D 360 playback.