Paper summary: InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

Problem

Most MLLMs start from a finished text-only LLM and adapt it to images afterwards, which brings alignment difficulties.

Method

InternVL3 uses native multimodal pre-training: multimodal and language abilities are learned together in a single pre-training stage, from multimodal data plus pure text. It adds:

• variable visual position encoding (V2PE), for longer multimodal contexts
• post-training with SFT and mixed preference optimisation (MPO), the method from the Nov 2024 paper on this blog
• test-time scaling strategies and optimised training infrastructure

Key results

From the abstract: InternVL3-78B scores 72.2 on MMMU, a new state of the art among open-source MLLMs, and remains competitive with ChatGPT-4o, Claude 3.5 Sonnet and Gemini 2.5 Pro while keeping strong language ability. Training data and weights are to be released.

Limitations

There’s no dedicated limitations section in the extracted text. Comparisons are the developers’ own.