Integrated Gradients: Sensitivity and Invariance

This article explores how the Integrated Gradients method attributes artificial intelligence outputs to specific input pixels while strictly adhering to the fundamental axioms of sensitivity and implementation invariance. Standard gradient-based attribution methods often fail to accurately credit input features due to saturation in neural networks, leading to misleading explanations of model behavior. By accumulating gradients along a straight-line path from a neutral baseline to the actual input, Integrated Gradients overcomes these limitations. The following sections break down the mechanics of this method and explain precisely how its mathematical foundation guarantees both sensitivity and implementation invariance.

Understanding Integrated Gradients

To understand how the axioms are satisfied, one must first understand the mechanism of Integrated Gradients. When an AI model processes an image, it assigns a prediction score based on the input pixels. Integrated Gradients calculates the importance of each pixel by comparing the actual image to a baseline image—typically a completely black image.

Instead of only looking at the gradients of the final image, the method generates a series of interpolated images that gradually transition from the black baseline to the actual image. It calculates the gradients of the model's output with respect to the pixels at each step along this straight-line path and integrates (sums) them. This accumulation of gradients measures the exact contribution of each pixel to the change in the model's prediction from the baseline to the final input.

Satisfying the Axiom of Sensitivity

The axiom of sensitivity states that if an input and a baseline differ in exactly one feature (such as a single pixel) and produce different predictions, that specific feature must be given a non-zero attribution.

Standard gradient methods frequently violate sensitivity. Neural networks use activation functions like ReLU or Sigmoid, which can flatten out—or saturate—when inputs reach certain thresholds. In these flat regions, the local gradient drops to zero. Even if a pixel was wholly responsible for pushing the network into that high-confidence region, a standard gradient check at the final input will yield zero, falsely implying the pixel is unimportant.

Integrated Gradients satisfies sensitivity by integrating gradients across the entire path from the baseline to the input. Because the model's prediction changes between the baseline and the final image, the fundamental theorem of calculus guarantees that the gradient cannot be zero at every point along the path. By capturing the gradients before the network saturates and summing them up, Integrated Gradients ensures that the influential pixel receives the non-zero attribution it deserves.

Satisfying Implementation Invariance

Two neural networks are considered functionally equivalent if they produce the exact same outputs for all possible inputs, even if their internal architectures, layers, and weights are arranged differently. The axiom of implementation invariance dictates that an attribution method must produce identical pixel attributions for any two functionally equivalent networks.

Many attribution methods fail this axiom because they rely on rules that modify backpropagation based on specific layer types or internal node activations. Consequently, if you rearrange the internal structure of a network without changing its input-output behavior, those methods yield different attributions.

Integrated Gradients satisfies implementation invariance because it relies solely on the standard mathematical gradients of the output with respect to the input. According to the chain rule of calculus, the gradient of a function depends only on the overall input-output mapping, not on how intermediate calculations are structured. Because Integrated Gradients only computes and sums these pure gradients along a defined path of inputs, the internal implementation details of the network are completely abstracted away. As long as two models behave identically on the inputs along the path, their gradients at every step will be identical, resulting in the exact same pixel attributions.