Paper summary: BabyVision: Visual Reasoning Beyond Language
BabyVision tests core visual abilities that don't depend on language, which young children handle easily; the best model tested (Gemini3-Pro-Preview) scores 49.7, behind 6-year-olds and far behind adults' 94.1.
Problem
Humans develop core visual skills long before language. MLLMs, the authors argue, lean on linguistic priors to make up for fragile visual understanding, and fail basic visual tasks even 3-year-olds solve. (BlindTest, summarised here in July 2024, found a related weakness.)
Method
BabyVision assesses core visual abilities independent of linguistic knowledge: 388 items in 22 subclasses across four categories, with human evaluation for baselines. The authors also explore solving visual reasoning with generation models through BabyVision-Gen and an automatic evaluation toolkit.
Key results
From the abstract: leading MLLMs score well below human baselines. Gemini3-Pro-Preview scores 49.7, behind 6-year-old humans and far behind the average adult score of 94.1. Code and data are released.
Limitations
There’s no dedicated limitations section in the extracted text. With 388 items, the benchmark is small, and it’s built to isolate one kind of weakness.
Sources
- BabyVision: Visual Reasoning Beyond Language (arXiv:2601.06521) — arXiv
- Hugging Face Daily Papers, 2026-01-13 — Hugging Face
- BabyVision: Visual Reasoning Beyond Language (arXiv HTML full text) — arXiv
- Code repository (linked in the paper) — GitHub
This is a summary written from the linked public sources. Figures and numbers are as reported there.