Paper summary
Paper summary: Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models
Molmo is a family of open vision-language models trained on PixMo, new datasets collected without using any external VLMs, including detailed captions, free-form Q&A and 2D pointing data.
- Many "open" VLMs are effectively distilled from proprietary VLMs through synthetic data; PixMo avoids that.
- A 2D pointing dataset teaches the model to point at things in images, not just describe them.