Facebook Launches Ego4D Project to Teach AI First-Person Perception
Facebook introduces Ego4D, a massive dataset of first-person videos designed to shift AI from third-person understanding to human-like egocentric perception. The project aims to unlock new possibilities for augmented and virtual reality experiences.
Facebook introduces Ego4D, a long-term research project that aims to teach artificial intelligence to understand the world from a first-person perspective. Unlike most AI systems that learn from third-person, omniscient views, this new approach focuses on egocentric perception to help AI interact with the world exactly like humans do.
The project involves a global consortium of universities and labs that collect over 2,200 hours of first-person video from more than 700 participants across 73 cities. Facebook Reality Labs also contributes an additional 400 hours of data using smartglasses in staged environments to further supplement this massive dataset.
According to Facebook researchers, current computer vision systems fail to recognize familiar objects when viewed from the center of the action, such as a camera on a rollercoaster. By tackling the challenges of embodied AI through this new paradigm of first-person perception, Facebook hopes to enable unprecedented augmented and virtual reality experiences.