Facebook SEER Model Proves Self-Supervised AI Can Beat Supervised Systems
Facebook AI introduces SEER, a massive self-supervised computer vision model that achieves state-of-the-art accuracy without relying on labeled data. The system outperforms traditional supervised models on key visual tasks using a billion random, unlabeled Instagram images.
Facebook AI researchers announce SEER, a billion-parameter self-supervised model that outperforms state-of-the-art self-supervised systems and traditional supervised models on tasks like image classification, object detection, and segmentation. Instead of relying on carefully labeled datasets, this system generates its own data labels by finding relationships between completely random, unannotated images. Experts consider this self-supervised learning approach a crucial step toward developing artificial intelligence with human-like common sense.
The model achieves 84.2 percent accuracy on the popular ImageNet dataset after pretraining on a billion public Instagram images, setting a new record for self-supervised systems. Even when researchers train SEER using only 10 percent or 1 percent of the ImageNet dataset, it still reaches nearly 78 percent and over 60 percent accuracy, respectively. This remarkable efficiency demonstrates that massive amounts of unlabeled data are highly effective for training robust computer vision models.
Facebook relies on powerful NVIDIA hardware to train this massive system, using a cluster of 512 NVIDIA V100 Tensor Core GPUs with 32GB of RAM for 30 days. The development team selects the RegNet architecture for its ability to scale to billions of parameters while respecting memory and runtime constraints. Additionally, the SwAV online clustering algorithm, combined with mixed precision tools from the NVIDIA Apex library and PyTorch gradient checkpointing, helps achieve record performance with six times less training time.