Meta Releases Llama 3.1 405B, Its Largest Open Source AI Model Yet
Meta introduces the Llama 3.1 model family, featuring a massive 405 billion-parameter open source AI system with a 128K context window. The release goes against the current industry trend of smaller models, though high deployment costs pose a challenge for some organizations.
Meta releases the Llama 3.1 family of large language models, headlined by the 405B variant that stands as the company's largest generative AI model to date. The updated model family also includes refreshed 70B and 8B versions, with all models benefiting from an expanded 128K context window that allows more information to flow through the AI system. Additionally, Meta supports eight languages and updates its license to permit developers to use Llama outputs to improve other models.
This release goes against the current AI marketplace trend that favors small language models. Analysts note that the Llama 3.1 405B is the first customizable open source large language model of this scale, giving companies a powerful new option to modify and deploy as they see fit. However, creating this massive model requires immense resources, with Meta using more than 16,000 Nvidia H100 GPUs that cost up to $40,000 each to train the system.
These steep computational requirements mean that deploying and maintaining the 405B model demands significant upfront hardware costs alongside ongoing expenses for electricity and cooling. As a result, the massive model remains too costly for some smaller enterprises to practically operate on their own. Despite these financial barriers, experts emphasize that having a large open source model available ultimately unlocks substantial value and flexibility for larger organizations seeking advanced AI capabilities.