Meta Releases Massive Open-Source Language Model to Promote AI Transparency
Meta breaks from Big Tech norms by freely releasing a fully trained large language model to researchers. The unprecedented move allows outside experts to examine the system's remarkable abilities and harmful flaws.
Meta's AI lab creates a massive new language model that shares the remarkable abilities and harmful flaws of OpenAI's GPT-3. In an unprecedented move for Big Tech, the company gives the fully trained model away for free to researchers, along with details about how it is built and trained. Joelle Pineau, managing director at Meta AI, emphasizes that inviting collaboration and scrutiny is an essential part of the research process.
This release marks the first time a fully trained large language model is available to any researcher who wants to study it. Experts like computational linguist Emily M. Bender and Hugging Face chief scientist Thomas Wolf praise the decision, noting that it breaks the trend of small teams developing powerful technology behind closed doors. The move provides ethicists and social scientists with access they previously lack.
Large language models generate highly convincing text but suffer from deep flaws like parroting misinformation, prejudice, and toxic language. Because these models require massive amounts of data and computing power, only wealthy tech firms have the resources to build them. Meta aims to bridge the gap between universities and industry, hoping that wider access leads to better understanding and improvement of the technology's underlying problems.