Facebook Trains Billion-Parameter Image AI Using One Billion Unlabeled Instagram Photos
Facebook develops SEER, a massive computer vision model that partly teaches itself by grouping one billion public Instagram images before fine-tuning on labeled data.
Facebook builds its most advanced semi-supervised computer vision system to date by training it on a massive dataset of one billion public Instagram photos. Known as SEER, this massive convolutional neural network contains over a billion parameters and accurately identifies and describes various objects in images, such as bicycles or umbrellas.
What makes SEER unique is its self-supervised training method, where the system initially teaches itself to group similar Instagram pictures together without any human intervention using an algorithm called SwAV. After this unsupervised pretraining phase, the development team fine-tunes the model using a traditional supervised approach by associating a million labeled ImageNet photos with their human-written text captions.
This semi-supervised approach proves far more efficient than manually labeling a billion Instagram images for traditional training. By combining unsupervised pretraining with supervised fine-tuning, Facebook follows a growing machine learning trend utilized by other massive AI models like OpenAI's GPT-3 to handle enormous datasets with minimal human labeling effort.