NeurIPS 2020 Researchers Propose Alternatives to Flawed Backpropagation
Researchers at NeurIPS 2020 introduce new alternatives to backpropagation that aim to solve the problem of catastrophic forgetting in AI models. These proposed continuous learning methods mimic human memory to allow neural networks to learn new information without losing past knowledge.
Backpropagation is a decades-old algorithm that trains neural networks by adjusting weights to minimize prediction errors, and it drives many modern advances in speech recognition and computer vision. However, this standard method forces AI models to optimize for a fixed target, which creates a major limitation when the models encounter new data.
This limitation leads to a problem known as "catastrophic forgetting," where a neural network loses its previously learned knowledge upon learning new information. To overcome this, researchers are developing continuous learning techniques that allow AI to learn continuously without needing to retrain on its entire historical dataset.
At the virtual NeurIPS 2020 conference, researchers propose dozens of alternatives to traditional backpropagation to enable this more humanlike learning. These innovative approaches range from leveraging specialized hardware like photonic circuits to adopting highly modular and flexible training methods.