DeepMind Pushes AI Boundaries With Chinchilla and Ithaca Models
DeepMind releases a flurry of research papers in 2022, highlighting breakthroughs like the compute-optimal Chinchilla language model and the Ithaca tool for restoring ancient texts.
DeepMind pushes the frontiers of artificial intelligence by publishing 34 research papers in just four months. The Alphabet subsidiary focuses on scaling models effectively and applying AI to unique historical challenges. These publications demonstrate the lab's ongoing commitment to advancing machine learning capabilities across various domains.
One standout paper reveals that model size and dataset size require equal scaling for compute-optimal training. DeepMind tests this theory by developing Chinchilla, a model trained with the same compute budget as Gopher but featuring 70 billion parameters and four times more data. Chinchilla outperforms massive competitors like GPT-3 and Gopher, achieving an average accuracy of 67.5% on the MMLU benchmark.
Another significant project introduces Ithaca, a deep neural network designed to restore and attribute ancient Greek inscriptions. The tool achieves 62% accuracy in repairing damaged texts independently, boosting historian accuracy from 25% to 72% when used as a decision support system. Additionally, Ithaca successfully maps these ancient writings to their original geographical locations with 71% accuracy.