FFmpeg RTMPS Streaming with Client Certificates

Streaming securely over RTMPS with mutual TLS (mTLS) requires passing client-side certificates and keys to FFmpeg to authenticate your stream against the destination server. This guide details how to verify your FFmpeg build for TLS support, format your authentication certificates, and execute the exact FFmpeg command needed to establish a secure RTMPS broadcast with client certificate authentication.

1. Verify FFmpeg Build Requirements

FFmpeg must be compiled with SSL/TLS support to handle encrypted RTMPS connections and mutual authentication. Run the following command to check your FFmpeg build configuration:

ffmpeg -version

Look for --enable-openssl or --enable-gnutls in the configuration output. If your build uses OpenSSL or GnuTLS, it supports the required TLS flags. If neither is present, you must install or compile an FFmpeg build that includes TLS support.

2. Prepare Certificate and Key Files

The server requires your identity in PEM format. Ensure you have the following files accessible on your local system:

  • Client Certificate: client.crt or client.pem (public certificate)
  • Client Private Key: client.key (decrypted RSA or ECC private key)
  • CA Certificate (Optional/Recommended): ca.crt (the Certificate Authority that signed the server's certificate, required if using self-signed or private enterprise certificates)

Ensure the client private key is not encrypted with a passphrase, as FFmpeg cannot interactively prompt for a password during headless execution. To remove a passphrase from an RSA private key using OpenSSL:

openssl rsa -in encrypted_client.key -out client.key

Set appropriate file permissions so FFmpeg can read the files:

chmod 600 client.key
chmod 644 client.crt

3. Configure and Run the FFmpeg Command

FFmpeg passes TLS options down to the underlying socket layer using either command-line options or query parameters appended to the RTMPS URL.

Place the TLS options directly before the output URL:

ffmpeg -re -i input.mp4 \
  -c:v libx264 -preset veryfast -b:v 3000k -maxrate 3000k -bufsize 6000k \
  -c:a aac -b:a 128k -ar 44100 \
  -f flv \
  -tls_cert "/path/to/client.crt" \
  -tls_key "/path/to/client.key" \
  -ca_file "/path/to/ca.crt" \
  -tls_verify 1 \
  "rtmps://rtmps.example.com:443/live/streamkey"
  • -tls_cert: Path to the client's public certificate.
  • -tls_key: Path to the client's private key.
  • -ca_file: Path to the trusted Certificate Authority file to validate the server.
  • -tls_verify 1: Enforces validation of the server certificate chain. Set to 0 only for debugging if certificate verification fails due to hostname mismatches.

Method B: URL Query Parameters

If your specific FFmpeg build does not parse the -tls_cert and -tls_key flags before the output format, pass them as URL-encoded query parameters within the RTMPS destination URL:

ffmpeg -re -i input.mp4 \
  -c:v copy -c:a aac \
  -f flv \
  "rtmps://rtmps.example.com:443/live/streamkey?tls_cert=/path/to/client.crt&tls_key=/path/to/client.key&ca_file=/path/to/ca.crt&tls_verify=1"

4. Troubleshooting Common Issues

  • Handshake Failure / unknown error: If the handshake terminates before data transmission, check FFmpeg’s verbose logs by adding -loglevel debug to your command. This will output the exact point of TLS failure.

  • Key/Certificate Mismatch: Verify that your public certificate matches your private key using OpenSSL:

    openssl x509 -noout -modulus -in client.crt | openssl md5
openssl rsa -noout -modulus -in client.key | openssl md5

    The output hashes must be identical.

  • Port Numbers: Ensure the destination URL explicitly uses port 443 (rtmps://example.com:443/...) if the server expects TLS traffic on standard HTTPS/RTMPS ports rather than the default plain RTMP port 1935.