Top research teams at NeurIPS 2020 showcase new methods for improving transformer efficiency, reducing gender bias, and advancing multilingual NLP capabilities. Pre-trained language models continue to dominate the conference's natural language processing track.