OpenAI Whisper Delivers Open-Source Multilingual Speech Recognition
OpenAI unveils Whisper, a 1.6 billion parameter AI model that transcribes and translates speech in 97 languages using weakly supervised web data.
OpenAI releases Whisper, a 1.6 billion parameter AI model that transcribes and translates speech audio across 97 different languages. The system relies on an encoder-decoder Transformer architecture to process audio in 30-second chunks and demonstrates impressive zero-shot performance without requiring fine-tuning on benchmark datasets.
Unlike traditional state-of-the-art automated speech recognition (ASR) models, Whisper learns from weakly supervised data instead of relying on transfer learning. The model trains on 680,000 hours of noisy audio and transcript pairs scraped directly from the internet, avoiding the common pitfalls of fine-tuned models that often fail to generalize well to new audio conditions.
In zero-shot evaluations, Whisper makes 55% fewer errors than the baseline Wav2Vec model, proving the effectiveness of this unique training approach. OpenAI open-sources the models and inference code to help developers easily build robust voice interfaces and drive further research into practical speech processing applications.