Synthetic Gradients

Synthetic gradients allow parts of a neural network to update their weights independently, without waiting for a full backpropagation pass through the entire network. This technique can speed up training and enable more asynchronous or parallelized learning in complex models.