Meta Unveils Chameleon, a Native Multimodal AI Competitor
Meta introduces Chameleon, a new family of AI models built from the ground up to natively process and generate mixed image and text sequences. The early-fusion architecture achieves state-of-the-art results on visual tasks while remaining competitive in text-only benchmarks.
Meta introduces Chameleon, a new family of AI models that uses a native, early-fusion approach to handle multiple types of data. Unlike popular late-fusion systems that stitch together separate models for text and images, Chameleon learns from an interleaved mixture of images, text, and code from the ground up.
The model works by converting images into discrete tokens and combining them with text and code tokens into a single, unified vocabulary. This end-to-end transformer architecture allows Chameleon to seamlessly process and generate sequences of interleaved images and text without needing modality-specific components.
Reported experiments show that Chameleon achieves state-of-the-art performance in visual question answering and image captioning while staying competitive in text-only tasks. This deeply integrated design rivals Google Gemini's early-fusion approach but stands out by removing the need for separate image decoders during the generation phase.