Google Unveils Meena, a Chatbot That Nears Human Conversation Skills
Google introduces Meena, a new chatbot trained on massive amounts of social media data that outperforms competitors in conversational tests. The company holds back a public release to thoroughly check the AI for safety and bias issues.
Google unveils a new neural-network-powered chatbot called Meena that it claims outperforms all other chatbots in the world. The AI learns from a massive dataset of 341 gigabytes of public social-media conversations, which is over eight times the amount of data used to train OpenAI's GPT-2. Because of this extensive training, Meena holds conversations on virtually any topic and even attempts to create its own jokes.
To measure Meena's abilities, Google creates a new metric called the Sensibleness and Specificity Average (SSA) that evaluates whether a response makes sense and relates directly to the previous statement. Unlike many chatbots that give vague replies to hide their lack of understanding, Meena provides highly specific and context-aware answers. In these tests, Meena achieves an impressive SSA score of 79%, far surpassing the previous leading chatbot Mitsuku which only scores 56%, and coming close to the 86% average of human conversationalists.
Despite the impressive results, the public cannot interact with Meena just yet. Google explicitly states that it refuses to release a public demo until researchers thoroughly vet the model for safety and bias, a necessary precaution to prevent the AI from generating harmful or offensive text. This careful approach directly addresses past failures in the industry, such as Microsoft's Tay chatbot, which notoriously began spewing toxic rhetoric within hours of its social media launch.