Facebook AI Develops SEER, a Self-Supervised Vision Model That Beats Supervised Rivals
Facebook AI introduces SEER, a self-supervised computer vision model that achieves record accuracy on ImageNet without relying on labeled data. The massive billion-parameter system trains on random public images to outperform existing state-of-the-art models.
Facebook AI researchers announce SEER, a massive self-supervised model that surpasses the best self-supervised systems and outperforms traditional supervised models on image classification, object detection, and segmentation tasks. Instead of relying on carefully labeled datasets, this billion-parameter model generates its own data labels by finding relationships between completely random, unannotated images. This approach brings AI closer to developing human-like common sense, according to Facebook AI chief scientist Yann LeCun.
The system achieves impressive results after pretraining on a billion random public Instagram images, hitting 84.2 percent accuracy on the popular ImageNet dataset. SEER also demonstrates remarkable efficiency, as it achieves nearly 78 percent accuracy using only 10 percent of the ImageNet dataset and maintains over 60 percent accuracy with just 1 percent of the data. This high performance with limited labeled data showcases the power of self-supervised learning for real-world applications where annotations are scarce.
Facebook trains SEER on a powerful array of 512 NVIDIA V100 Tensor Core GPUs over a period of 30 days. The engineering team utilizes mixed precision from the NVIDIA Apex library and gradient checkpointing tools from PyTorch to optimize memory usage and increase training speed. By combining the highly scalable RegNet architecture with the SwAV online clustering algorithm, the researchers successfully reduce training time by six times while setting new performance records.