GIMP Feather Before vs After Shrink Selection

In GIMP, modifying a selection via feathering and shrinking involves two fundamentally different mathematical transformations: linear spatial convolution and non-linear morphological erosion. Because these mathematical operations are non-commutative, executing them in different orders yields distinctly different selection masks. Feathering before shrinking creates an asymmetric, clipped gradient with a compressed dynamic range, whereas shrinking before feathering produces a geometrically uniform, symmetric transition shifted inward.

Selection Mask Representation in GIMP

GIMP stores selections as an 8-bit grayscale raster channel known as a selection mask, where each coordinate \((x, y)\) has an intensity value \(S(x, y) \in [0, 255]\) (or normalized to \([0.0, 1.0]\)). A value of \(1.0\) represents a fully selected pixel, \(0.0\) represents an unselected pixel, and intermediate values represent fractional selections.

Mathematical Definitions of the Operations

  1. Shrink Selection (Morphological Erosion): Shrinking a selection by a radius \(r\) mathematically corresponds to grayscale morphological erosion (\(\ominus\)) using a circular structuring element \(B_r\). For each pixel \((x, y)\), the operation evaluates the local neighborhood and selects the infimum (minimum value): \[(S \ominus B_r)(x, y) = \inf_{(u, v) \in B_r} S(x + u, y + v)\] On a hard, binary selection, this strictly contracts the boundary by Euclidean distance \(r\).

  2. Feather Selection (Gaussian Convolution): Feathering applies a linear low-pass filter, mathematically defined as the spatial 2D convolution (\(*\)) of the selection mask with a normalized Gaussian kernel \(G_\sigma\): \[(S * G_\sigma)(x, y) = \iint S(u, v) G_\sigma(x - u, y - v) \, du \, dv\] This transforms step-function boundaries into smooth, continuous sigmoidal transitions following the Gaussian cumulative distribution function.

The Order of Operations

Because morphological erosion is a non-linear operation and convolution is a linear operation, they do not commute:

\[(S * G_\sigma) \ominus B_r \neq (S \ominus B_r) * G_\sigma\]

1. Shrink First, Then Feather: \((S \ominus B_r) * G_\sigma\)

When the selection is shrunk first, the morphological erosion acts upon a binary step function (values of strictly \(0\) or \(1\)). The boundary moves uniformly inward by \(r\) pixels while remaining a sharp step edge.

Applying the feather operation afterward convolves this shifted step edge with the Gaussian kernel. The resulting edge profile is a mathematically pure Gaussian blur centered exactly along the newly shrunken perimeter. The transition across the edge is symmetric, smoothly moving from \(1.0\) to \(0.0\) across the defined feather radius, preserving the intended falloff curve.

2. Feather First, Then Shrink: \((S * G_\sigma) \ominus B_r\)

When the selection is feathered first, the sharp edge becomes an extended gradient of intermediate floating-point values between \(0.0\) and \(1.0\).

Applying morphological erosion to this continuous gradient forces the algorithm to take the minimum intensity value within the structuring element \(B_r\) for every point. This produces several mathematical distortions:

  • Gradient Asymmetry: Morphological erosion pulls the lowest neighborhood value into the center of the kernel. This skews the sigmoid curve, causing the outer values of the gradient to decay toward zero much faster than normal Gaussian dispersion.
  • Loss of Sub-pixel Softness: By taking the minimum value across radius \(r\), the higher selection values near the inner edge are aggressively reduced, flattening the apex of the curve.
  • Shift of the 50% Threshold: In an erosion-after-blur pipeline, the \(0.5\) opacity contour does not move inward by precisely \(r\). Instead, the shift magnitude depends directly on the slope (variance \(\sigma\)) of the existing blur, leading to an irregular geometric contraction compared to a binary shrink.

Shrinking before feathering ensures linear consistency, geometric accuracy, and a symmetric Gaussian profile. Feathering before shrinking subjects continuous gradient values to non-linear minimization, steepening the falloff and distorting the selection profile.