Stream FFmpeg HLS Directly to Amazon S3

This guide explains how to configure FFmpeg to output HTTP Live Streaming (HLS) segments and playlists directly to an Amazon S3 bucket. Because FFmpeg does not natively support the AWS S3 API authentication protocols, direct streaming requires using either a mounted S3 file system, a real-time sync utility, or an intermediate HTTP proxy. Below, you will find the step-by-step configurations for the most reliable methods to achieve this pipeline.

Method 1: Using an S3 FUSE Mount (Recommended for Ease)

The most straightforward way to write FFmpeg output directly to Amazon S3 is by mounting your S3 bucket as a local directory on your server using tools like Goofys or s3fs-fuse. This allows FFmpeg to write HLS segments (.ts) and playlists (.m3u8) as if it were writing to a local hard drive.

Step 1: Install and Mount S3FS

1. Install s3fs on your system (e.g., Ubuntu/Debian):

   sudo apt-get install s3fs

2. Save your AWS credentials in /etc/passwd-s3fs or ~/.passwd-s3fs:

   echo "ACCESS_KEY_ID:SECRET_ACCESS_KEY" > ~/.passwd-s3fs
   chmod 600 ~/.passwd-s3fs

3. Create a mount point and mount your S3 bucket:

   mkdir ~/s3-bucket
   s3fs your-bucket-name ~/s3-bucket -o passwd_file=$HOME/.passwd-s3fs -o allow_other

Step 2: Run the FFmpeg Command

Point your FFmpeg output path to the mounted directory.

ffmpeg -i input.mp4 \
  -codec:v libx264 -codec:a aac \
  -map 0 \
  -f hls \
  -hls_time 10 \
  -hls_list_size 0 \
  -hls_segment_filename '~/s3-bucket/stream_%03d.ts' \
  ~/s3-bucket/playlist.m3u8

Method 2: Local Spooling with Real-Time AWS CLI Sync (Recommended for Production)

Writing directly to S3 via FUSE mounts can sometimes suffer from latency issues because FFmpeg continuously updates the .m3u8 playlist file. To prevent encoding lag, you can write files locally and upload them to S3 in real-time using a background process.

Step 1: Run FFmpeg writing to a local folder

ffmpeg -i input.mp4 \
  -codec:v libx264 -codec:a aac \
  -f hls \
  -hls_time 6 \
  -hls_list_size 10 \
  -hls_flags delete_segments \
  -hls_segment_filename '/tmp/hls/stream_%03d.ts' \
  /tmp/hls/playlist.m3u8

(Note: -hls_flags delete_segments keeps the local disk clean if you are streaming live).

Step 2: Sync to S3 continuously

In a separate terminal or script, run the AWS CLI sync command in a loop to push new segments to your bucket instantly:

while true; do
  aws s3 sync /tmp/hls/ s3://your-bucket-name/hls/ --acl public-read
  sleep 2
done

Method 3: Using Direct HTTP PUT (Advanced)

FFmpeg’s HLS muxer can upload segments directly via HTTP POST or PUT requests. However, because Amazon S3 requires AWS Signature Version 4 authentication headers, you cannot send the stream directly to S3 without an intermediate proxy (like an Nginx server or an API Gateway) that handles the authentication.

If your proxy is configured to sign and forward requests to S3, use the following FFmpeg configuration:

ffmpeg -i input.mp4 \
  -codec:v libx264 -codec:a aac \
  -f hls \
  -hls_time 10 \
  -method PUT \
  -hls_segment_filename 'http://your-auth-proxy.com/hls/stream_%03d.ts' \
  http://your-auth-proxy.com/hls/playlist.m3u8