DeepMind Unveils Gopher, a Massive 280-Billion Parameter Language Model
DeepMind introduces Gopher, a 280 billion parameter language model, alongside new research on the ethical risks and architectural efficiencies of large-scale AI. The studies reveal that while larger models excel at fact-checking and reading comprehension, they still struggle with logical reasoning.
DeepMind introduces Gopher, a massive 280 billion parameter transformer language model designed to advance the development of safe and efficient artificial intelligence. As part of a broader push to understand language processing, researchers train a series of models ranging from 44 million to 280 billion parameters to see how scale impacts performance.
The research reveals that increasing the size of a language model significantly boosts its abilities in specific areas like reading comprehension, fact-checking, and the identification of toxic language. However, the study also highlights that simply scaling up a model does not automatically improve its performance in logical reasoning and common-sense tasks.
Alongside the technical release of Gopher, DeepMind publishes complementary studies focusing on the ethical and social risks associated with large language models and a new architecture that offers better training efficiency. This interdisciplinary approach emphasizes the importance of collaborating with diverse experts to anticipate and mitigate the challenges of training algorithms on existing human datasets.