Backpropagation

The algorithm for computing gradients of a loss function with respect to neural network parameters by applying the chain rule layer by layer from output to input. Backpropagation enables gradient-based optimization of deep networks. It requires storing intermediate activations during the forward pass, which dominates memory consumption during training.