Visualize H.264 Macroblocks with FFmpeg Codecview

This article explains how to use FFmpeg’s codecview filter to visualize H.264 macroblock types and motion vectors during video playback. By utilizing ffplay alongside specific decoder flags, you can overlay analytical data directly onto your video stream to inspect how frames are partitioned and compressed.

To visualize macroblock types during playback, you must use ffplay (FFmpeg’s media player) and instruct the video decoder to export the motion vectors and block graphics.

Run the following command in your terminal:

ffplay -flags2 +export_mvs -vf "codecview=block_type=mbt" input.mp4

Command Breakdown

• ffplay: The playback tool utilized to render the video and filter effects in real time.
• -flags2 +export_mvs: This flag forces the H.264 decoder to export motion vector and macroblock side data, which is required for the visualization filter to function.
• -vf "codecview=block_type=mbt": This applies the video filter. The block_type (or bt) option is set to mbt (macroblock type) to color-code the different block types.

Visualizing Macroblocks and Motion Vectors Together

If you want to analyze both the macroblock types and the motion vectors (which show how blocks move from frame to frame) simultaneously, you can combine the parameters:

ffplay -flags2 +export_mvs -vf "codecview=block_type=mbt:mv=pf+bf+bb" input.mp4

In this command, the mv parameter is configured with: * pf: Forward predicted P-frame motion vectors (rendered in green). * bf: Forward predicted B-frame motion vectors (rendered in blue). * bb: Backward predicted B-frame motion vectors (rendered in red).

Understanding the Visual Output

When the video plays with the block_type=mbt filter active, different types of macroblocks are overlaid with distinct colors:

• Intra-coded blocks (I-blocks): Usually represented in red. These blocks are self-contained and do not rely on other frames for temporal prediction.
• Inter-coded blocks (P-blocks and B-blocks): Usually represented in green or blue. These blocks rely on motion estimation from past or future frames.
• Skip blocks: Uncolored or marked differently, representing areas of the frame that have not changed from the previous frame.