Paper summary
Paper summary: Chameleon: Mixed-Modal Early-Fusion Foundation Models
Chameleon is one token-based model that reads and generates images and text in any order, using early fusion, plus a training recipe designed to keep this mixed-modal setup stable.
- Images are turned into discrete tokens and mixed with text tokens from the very first layer, which is what "early fusion" means.
- Human judges rated its long-form mixed image-and-text outputs as matching or beating much larger models.