Visualize H.264 Macroblocks with FFmpeg Codecview

This guide explains how to use FFmpeg's codecview filter and the ffplay utility to inspect H.264 macroblock types directly during video playback. By rendering a color-coded visual overlay of intra- and inter-coded blocks, video engineers and analysts can evaluate encoder decisions, prediction modes, and macroblock partitioning in real time.

Running the FFplay Command

To display macroblock types during playback, run the following command in your terminal:

ffplay -debug mb_type -vf "codecview=block_type=intra+inter" input.mp4

Parameter Breakdown

  • ffplay: The built-in media player included with the FFmpeg suite.
  • -debug mb_type: Instructs the H.264 software decoder to output macroblock type information to the video filter pipeline. Without this flag, the decoder may discard internal macroblock metadata.
  • -vf "codecview=...": Applies the video filter responsible for drawing the visual analysis overlays onto the video frames.
  • block_type=intra+inter: Configures codecview to highlight both Intra-coded macroblocks (I-blocks) and Inter-coded macroblocks (P- and B-blocks). You can also use the shorthand bt=intra+inter or bt=all.

Understanding the Visual Overlay

When enabled, the codecview filter outlines macroblocks and colors them based on their prediction method:

  • Intra Blocks (Red/Pink shades): These blocks are predicted entirely from adjacent pixels within the current frame, typical of I-frames or complex scenes in P/B-frames where temporal prediction fails.
  • Inter P-Blocks (Green shades): Forward-predicted blocks referencing previously decoded frames. Sub-partitions (e.g., 16x8, 8x16, 8x8) will be outlined to show how the encoder segmented the 16x16 macroblock.
  • Inter B-Blocks (Blue shades): Bi-directionally predicted blocks referencing both past and future frames.

Visualizing Macroblocks Alongside Motion Vectors

To gain deeper insight into how the encoder moves and predicts data, you can combine macroblock visualization with motion vector tracking:

ffplay -flags2 +export_mvs -debug mb_type -vf "codecview=block_type=intra+inter:mv=pf+bf+bb" input.mp4
  • -flags2 +export_mvs: Forces the decoder to export motion vector structures.
  • mv=pf+bf+bb: Visualizes forward-predicted P-frame vectors (pf), forward-predicted B-frame vectors (bf), and backward-predicted B-frame vectors (bb) as overlaid arrows originating from each block's center.

Technical Considerations

  • Software Decoding Only: Hardware acceleration (such as NVDEC, VAAPI, or VideoToolbox) bypasses standard software decoding pipelines and will not expose macroblock metadata. Ensure no hardware decoding flags (-hwaccel) are active.
  • H.264 Support: The mb_type export mechanism relies on FFmpeg’s native H.264 decoder. Modern codecs such as H.265 (HEVC) or AV1 use dynamic coding tree units (CTUs) and blocks of varying sizes rather than standard macroblocks; their visualization through codecview is limited primarily to motion vectors rather than complete block-type maps.