Paper summary: Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
InternVL 2.5 keeps InternVL 2.0's architecture but improves training, testing and data, and studies scaling of vision encoders, LLMs, data and test-time settings; it's reported to rival GPT-4o and Claude-3.5-Sonnet.
Problem
Open multimodal models trailed commercial ones. Which levers matter most when scaling: the vision encoder, the language model, the data or test-time settings?
Method
InternVL 2.5 keeps the InternVL 2.0 architecture and improves the training and testing strategies and data quality. The authors systematically explore performance trends as they scale vision encoders, language models, dataset sizes and test-time configurations.
Key results
From the abstract: on a wide range of benchmarks (multi-discipline reasoning, document understanding, multi-image and video, real-world comprehension, hallucination detection, visual grounding, multilingual, pure language), InternVL 2.5 is competitive with GPT-4o and Claude-3.5-Sonnet. It’s described as the first open-source MLLM to exceed 70% on MMMU, with chain-of-thought reasoning adding 3.7 points.
Limitations
There’s no dedicated limitations section in the extracted text. Like other developer-reported benchmark results, the comparisons are the authors’ own.
#multimodal#open-source#scaling
Sources
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (arXiv:2412.05271) — arXiv
- Hugging Face Daily Papers, 2024-12-09 — Hugging Face
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling (arXiv HTML full text) — arXiv
This is a summary written from the linked public sources. Figures and numbers are as reported there.