Paper summary
Paper summary: Vision language models are blind: Failing to translate detailed visual features into words
BlindTest's seven simple visual tasks, such as whether two circles overlap or how many times two lines cross, trip up strong vision-language models: four state-of-the-art VLMs averaged 58.07%.
- VLMs that ace vision benchmarks struggle when shapes overlap or sit close together.
- Linear probes show the vision encoder contains the needed information, so the bottleneck is turning it into words.