How to Create Cartoon Effect in FFmpeg

This article demonstrates how to transform standard video footage into an animated cartoon style using FFmpeg. By combining edge detection and blurring filters through FFmpeg’s filtergraph, you can smooth out video details while drawing bold outlines to achieve a classic cel-shaded cartoon aesthetic.

The Cartoon Effect Logic

To achieve a cartoon or cel-shaded look, the video processing is split into two parallel paths that are merged at the end: 1. The Color Path: Uses a blurring filter to smooth out gradients, remove noise, and create flat areas of color. 2. The Outline Path: Uses an edge detection filter to find boundaries, which are then inverted to create black outlines on a white background.

These two paths are combined using a multiplication blend, which overlays the black outlines onto the smoothed color video.

The FFmpeg Command

Run the following command in your terminal to apply the cartoon effect:

ffmpeg -i input.mp4 -vf "split=2 [color_in][edge_in]; [color_in] smartblur=lr=5:ls=-1.0:cr=3:cs=-1.0 [color_out]; [edge_in] edgedetect=low=0.05:high=0.1, negate [edge_out]; [color_out][edge_out] blend=all_mode=multiply" -c:a copy output.mp4

Command Breakdown

• split=2 [color_in][edge_in]: Splits the input video stream into two identical, temporary streams so we can process colors and edges separately.
• smartblur=lr=5:ls=-1.0:cr=3:cs=-1.0: Applies a detail-preserving blur to the color stream. The smartblur filter is ideal for cartoons because it smooths flat areas (reducing gradients to solid blocks of color) while keeping the actual edges relatively sharp.
• edgedetect=low=0.05:high=0.1: Applies a Canny edge detection algorithm to the second stream. This highlights the edges of objects in white on a black background.
• negate: Inverts the black-and-white edge map, turning the edges black and the background white.
• blend=all_mode=multiply: Multiplies the pixels of the smoothed color video with the inverted edge map. Because white pixels have a value of 1 and black pixels have a value of 0, the color video remains unchanged everywhere except where the black outlines are drawn.
• -c:a copy: Copies the original audio stream without re-encoding to save processing time.

Fine-Tuning the Results

You can customize the intensity and style of the cartoon effect by adjusting the filter parameters:

• Adjusting Outline Sensitivity: Change the low and high thresholds in edgedetect. Lower values (e.g., low=0.02:high=0.05) will detect more subtle lines, creating a highly detailed sketch look. Higher values will limit outlines only to major contrast boundaries.
• Adjusting Color Smoothness: Increase the radius (lr for luma radius, cr for chroma radius) in smartblur to make the color blocks larger and less detailed. For example, smartblur=lr=8:ls=-1.0:cr=5 will result in a more abstract, painted look.