Paper summary: Depth Anything V2

Problem

Monocular depth estimation (depth from a single image) needs training data, and real-world depth labels are noisy. Sensors fail on mirrors, glass and thin structures. Strong recent models built on Stable Diffusion were accurate but slow.

Method

The authors focus on data rather than “fancy techniques”, with three practices:

1. Replace all labelled real images with synthetic images, whose depth labels are precise.
2. Scale up the teacher model’s capacity.
3. Train student models through a “bridge” of large-scale pseudo-labelled real images: the teacher labels real photos, which closes the synthetic-to-real gap.

They release models from 25M to 1.3B parameters, and fine-tune them with metric depth labels to get metric-depth versions.

Key results

The abstract reports much finer and more robust predictions than V1, and more than 10× faster and more accurate results than the latest Stable-Diffusion-based models.

Limitations

The paper’s limitations discussion is mostly about evaluation. Common test sets (e.g. NYU-D, KITTI) have noisy labels on mirrors and thin structures, limited scene diversity and low resolution, so reported metrics aren’t fully reliable. That’s why the authors build a new benchmark with precise annotations and diverse scenes.