Batch Normalization
Batch Normalization is a technique used in training deep neural networks to normalize the inputs to each layer for a mini-batch. Instead of normalizing the entire dataset once, it normalizes each batch's activations, centering them around a mean of zero and scaling them to a standard deviation of one. This helps stabilize and accelerate the training process by reducing internal covariate shift, where the distribution of each layer's input changes during training.
Imagine a conveyor belt where items are constantly being produced, Batch Normalization is like a quality control station that ensures every new batch of items meets consistent size and weight standards before moving to the next stage.
It enables faster training of deeper networks and often acts as a regularizer, reducing the need for other regularization techniques.
Learn one new AI thing every day.
Daily Deck sends you seven plain-English cards like this every morning. Free.
Start free