FFmpeg Color Range Handling in MP4 Demuxing

When demuxing MP4 files, FFmpeg identifies and processes color range metadata to distinguish between limited (MPEG) and full (JPEG) color ranges. This article explains how FFmpeg extracts this information from both the MP4 container atoms and the underlying video bitstream, how it resolves potential metadata conflicts, and how it propagates this data to downstream decoders and filters to ensure accurate color reproduction.

Understanding MPEG vs. JPEG Range

Color range defines the span of digital values used to represent color and luminance in a video file:

• MPEG Range (Limited/TV Range): Standard for broadcast and most video distribution. In 8-bit video, luminance (Y) is restricted to values between 16 and 235, while chroma (Cb/Cr) ranges from 16 to 240.
• JPEG Range (Full/PC Range): Standard for computer graphics and photography. It utilizes the entire 8-bit spectrum from 0 to 255 for both luminance and chrominance.

If FFmpeg misidentifies the color range during demuxing, the resulting playback or transcode will suffer from visual defects: full-range content treated as limited-range will look overly contrasty with crushed shadows, while limited-range content treated as full-range will appear washed out.

How FFmpeg Detects Color Range During Demuxing

FFmpeg’s demuxer (libavformat) and decoder (libavcodec) work in tandem to determine the correct color range from an MP4 file. This detection happens at two distinct levels: the container level and the bitstream level.

1. Container-Level Extraction (MP4 Atoms)

The MP4 container format (ISO/IEC 14496-12) can store color space metadata within the track sample description box (stsd). Specifically, FFmpeg looks for the colr (Color Parameter) box inside the visual sample entry.

During the demuxing phase, libavformat parses the colr box to read three primary parameters, alongside a flag that specifies whether the video uses full or limited range. If this box is present, FFmpeg maps the parsed range to the stream’s codec parameters.

2. Bitstream-Level Extraction (Codec VUI)

The video bitstream itself (such as H.264/AVC or H.265/HEVC) contains its own metadata. Within the Sequence Parameter Set (SPS) of the video bitstream, there is a section called Video Usability Information (VUI).

The VUI contains a specific flag: video_full_range_flag. * If video_full_range_flag is set to 0, the range is MPEG (limited). * If video_full_range_flag is set to 1, the range is JPEG (full).

As FFmpeg demuxes the packets and passes them to the decoder, libavcodec parses this SPS VUI data to determine the range.

Resolving Container vs. Bitstream Conflicts

It is common for MP4 files to have conflicting metadata—for example, the MP4 container’s colr box might flag the video as limited range, while the internal H.264 bitstream flags it as full range.

FFmpeg handles these discrepancies through a prioritized resolution hierarchy:

1. Bitstream Precedence: By default, FFmpeg prioritizes the bitstream VUI parameters parsed by the decoder over the container-level metadata. The decoder’s findings directly update the AVFrame properties.
2. Internal API Mapping: The resolved range is mapped to the AVColorRange enum in FFmpeg’s API:
   • AVCOL_RANGE_MPEG (Limited range)
   • AVCOL_RANGE_JPEG (Full range)
   • AVCOL_RANGE_UNSPECIFIED (Fallback when no metadata is found; usually defaults to limited range for YUV formats)

Propagation to Downstream Filters (libswscale)

Once the demuxer and decoder establish the color range, this metadata is attached to each decoded frame (AVFrame->color_range).

When you apply filters or convert the pixel format (e.g., converting YUV420P to RGB for playback), FFmpeg’s scaling library, libswscale, reads this frame metadata. It selects the appropriate mathematical matrix to scale the color values. If the frame is tagged as AVCOL_RANGE_JPEG, libswscale uses a full-range conversion matrix; if tagged as AVCOL_RANGE_MPEG, it applies a limited-range correction matrix to prevent color shifting.