Study
Generative AI · Multimodal models (CLIP)
CLIP learns a shared space for images and text by contrastive training on image–caption pairs; zero-shot classification of photos by comparing them with text prompts.
- CLIP trains an image encoder and a text encoder so that matching image–caption pairs end up close together.
- Zero-shot classification is just "which text prompt is nearest to this image?".