Alexa AI Expands Research in Privacy and Fairness for Machine Learning
Alexa AI highlights its 2021 progress in trustworthy machine learning, focusing on protecting customer data and eliminating biases found in standard language models. The team explores differential privacy, federated learning, and new methods to mitigate gender disparities in AI systems.
Alexa AI prioritizes trustworthy machine learning by focusing heavily on privacy protection and algorithmic fairness. The research team addresses the significant challenge of eliminating inherent biases found in off-the-shelf language models like GPT-3 and RoBERTa, which often reflect the prejudices present in their public training texts. This dual focus on securing customer data and ensuring equitable performance forms the foundation of their latest AI advancements.
In the realm of privacy, the team explores differential privacy to rigorously quantify and protect model security against known vulnerabilities. They also advance federated learning, a distributed training method that keeps raw customer data entirely on-device by sending only model parameter updates to the cloud. This industrial-scale approach allows Alexa to learn from user interactions without compromising sensitive personal information.
To tackle fairness issues, Alexa AI introduces new measures to quantify and mitigate bias within machine learning models. One notable technique involves using counterfactual role reversal, where binary genders in real-world training examples are swapped to reduce gender disparity in distilled language models. The team presents these comprehensive findings at major upcoming industry conferences, including ACL, ICASSP, NAACL, and Interspeech.