Meta Unveils Llama 3.1 405B as Largest Open-Source AI Model
Meta releases Llama 3.1 405B, a massive 405-billion-parameter open-source language model. Major cloud providers immediately announce support for the new system.
Meta unveils Llama 3.1 405B, the largest open-source AI model to date with 405 billion parameters. This massive system is the flagship of the new Llama 3.1 family, which also includes smaller 8B and 70B versions. Meta achieves this by focusing heavily on data curation, scale, and managing complexity during the training process on 16,000 GPUs.
Major cloud vendors quickly announce their support for deploying the new model. Companies like Amazon, Microsoft, Databricks, Nvidia, IBM, and Snowflake offer ways for developers to build applications with Llama 3.1 405B. Amazon provides access through Bedrock, while Microsoft delivers the model via Azure AI's serverless API endpoint.
The new models feature an impressive 128,000-token context window that allows users to input hundreds of pages of text at once. They also support eight languages and include built-in tools for web search, math reasoning, and code execution. Meta significantly improves both the quantity and quality of the data used for pre-training and post-training to boost overall performance.