Mistral Releases Mixtral 8x7B, a Fast Open-Weight AI Model
Mistral AI unveils Mixtral 8x7B, a sparse mixture-of-experts model that outperforms Llama 2 70B while running six times faster. The open-weight model matches GPT-3.5 on most benchmarks at a fraction of the cost.
Mistral AI releases Mixtral 8x7B, a high-quality sparse mixture-of-experts model with open weights under the Apache 2.0 license. This new model outperforms Llama 2 70B on most standard benchmarks while offering inference that is six times faster. It stands as the strongest open-weight model with a permissive license, providing an unmatched cost-to-performance trade-off for developers.
The architecture relies on a decoder-only network where a router selects two out of eight distinct parameter groups to process each token. Although Mixtral contains 46.7 billion total parameters, it only utilizes 12.9 billion parameters per token. This approach allows the model to process input and generate output at the exact speed and cost of a much smaller 12.9 billion parameter model.
Mixtral matches or exceeds GPT-3.5 across most benchmarks and handles a context window of 32,000 tokens. It also supports multiple languages including English, French, Italian, German, and Spanish, while showing strong capabilities in code generation. When fine-tuned for instruction following, the model achieves an impressive score of 8.3 on the MT-Bench evaluation.