False as stated: convolution layers are translation equivariant, and pooling makes a CNN only approximately invariant to small shifts.
Proposed by @svpino
Card 1 of 259. Answer: False as stated: convolution layers are translation equivariant, and pooling makes a CNN only approximately invariant to small shifts.
About · Quizzes · History of AI · To suggest new questions or report an error send me a dm.
False. Convolutional layers are translation equivariant, not invariant: shift the input and the feature map shifts by the same amount, whereas invariance would mean the output does not change at all. A whole CNN is only approximately invariant to small shifts, mostly thanks to pooling.
Example in one dimension: the input [0, 0, 5, 0, 0, 0] with kernel [1, 2, 1] gives the feature map [5, 10, 5, 0]. Shift the input one step right and the map becomes [0, 5, 10, 5], the same values shifted. A global max gives 10 for both, which is invariant, but max pooling with width 2 and stride 2 gives [10, 5] and [5, 10], which differ.
This matters in practice. Azulay and Weiss showed in 2019 that shifting an image by a few pixels can change the predictions of standard ImageNet CNNs, and Richard Zhang's 2019 anti-aliased CNNs blur before downsampling to reduce the effect. Random crops and shifts in data augmentation teach the invariance that the architecture only approximates.
