Google BERT Revolutionizes Natural Language Processing With Record-Breaking Results
Google introduces BERT, a deep bidirectional Transformer model that achieves state-of-the-art results across 11 NLP tasks. The massive model even surpasses human performance in question answering benchmarks.
Google researchers unveil BERT, a groundbreaking deep bidirectional Transformer model that earns widespread praise from the machine learning community. This new architecture redefines state-of-the-art performance across 11 natural language processing tasks and notably surpasses human baselines in the challenging domain of question answering.
The BERT model stands out because it reads text in both directions simultaneously, unlike previous models such as OpenAI GPT or ELMo. Researchers train this massive architecture, which features 24 Transformer blocks and 340 million parameters, on a massive 3.3 billion word corpus using 16 TPU pods.
A key innovation in the BERT training process is the Masked Language Model technique, which randomly hides 15 percent of input tokens to force the system to learn deep bidirectional representations. The model also uses a next sentence prediction task to grasp vital relationships between sentences for complex applications like natural language inferencing.