Meta Releases 175-Billion Parameter Language Model to Researchers
Meta shares its massive OPT-175B language model with academics to help study and reduce AI bias and toxicity. The release includes the pretrained model and training code for researchers who lack the resources to build such systems from scratch.
Meta releases a massive language model known as OPT-175B to academic researchers to help them study and mitigate AI bias and toxicity. This model contains 175 billion parameters, matching the scale of commercial systems like OpenAI's GPT-3, and enables capabilities such as automated copywriting and coding assistance.
Proprietary tools remain out of reach for many academics due to a lack of access to underlying code and the massive computing resources required to train them. By providing both the pretrained models and the necessary code, Meta allows researchers to investigate these foundational technologies without needing hundreds of GPUs.
Meta releases smaller subsets of up to 66 billion parameters for anyone to use, while the full OPT-175B system is available only upon request for noncommercial research. The company notes that training such a massive system is extremely difficult, as its own team experiences numerous failures and restarts the process 35 times before achieving success.