FFmpeg Color Range Handling in MP4 Demuxing

During MP4 demuxing, FFmpeg identifies and extracts color range metadata—distinguishing between MPEG (limited/TV) range and JPEG (full/PC) range—by reading container-level atoms and stream-level bitstream parameters. This article explains how FFmpeg's libavformat parses the MP4 container, maps the range metadata to internal structures, resolves conflicts with elementary stream headers, and passes this information downstream to decoders and filters.

In digital video processing, color range defines the quantization levels used for luma and chroma samples. MPEG range (limited) restricts 8-bit luma values to 16–235 and chroma to 16–240, which is standard for broadcast and standard dynamic range (SDR) video. JPEG range (full) utilizes the entire 0–255 range for both luma and chroma, standard in still imaging and many computer-generated content pipelines. Within FFmpeg's codebase, these states are represented by the AVColorRange enumeration as AVCOL_RANGE_MPEG and AVCOL_RANGE_JPEG, alongside AVCOL_RANGE_UNSPECIFIED.

When demuxing an MP4 file, the FFmpeg mov demuxer (libavformat/mov.c) reads the ISO Base Media File Format box structure. Container-level color metadata is stored inside the track box (trak) under the sample table (stbl), within the sample description (stsd) entry for the video track. FFmpeg searches for a colr (Color Information) box. If the color type is specified as nclx (on-screen color parameters), the demuxer extracts the primary components: color primaries, transfer characteristics, matrix coefficients, and a 1-bit full_range_flag. If this flag is set to 1, FFmpeg assigns AVCOL_RANGE_JPEG to AVCodecParameters->color_range; if set to 0, it assigns AVCOL_RANGE_MPEG.

MP4 files often contain elementary video bitstreams (such as H.264/AVC or H.265/HEVC) that carry their own internal metadata within Sequence Parameter Sets (SPS) and Video Usability Information (VUI). The VUI contains its own video_full_range_flag. During demuxing, FFmpeg extracts packet data without fully decoding the video, but its internal parsers inspect the bitstream headers.

If a discrepancy exists between the container's colr box and the bitstream's VUI flag, FFmpeg prioritizes the container-level information at the demuxer stage (AVStream->codecpar). However, once packets are sent to the decoder (libavcodec), the decoder initializes context directly from the bitstream's VUI. If the bitstream explicitly states a color range, the decoder context (AVCodecContext->color_range) will typically overwrite the container setting for subsequent frame extraction.

If neither the colr box nor the bitstream VUI specifies a range, FFmpeg marks the property as AVCOL_RANGE_UNSPECIFIED. In the absence of defined metadata, downstream modules—such as libswscale or specific video encoders—apply standard fallbacks, typically assuming MPEG range for YUV color spaces and JPEG range for RGB color spaces to prevent crushing blacks or clipping highlights during rendering and transcoding.