Paper summary
Paper summary: Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization
Mixed Preference Optimization and the automatically built MMPR dataset improve chain-of-thought reasoning in multimodal LLMs (InternVL2-8B and 76B) beyond what supervised fine-tuning alone gives.
- Pre-training plus SFT leaves a distribution shift that especially hurts multimodal chain-of-thought.
- Preference optimisation on automatically built reasoning preferences helps close that gap.