Paper summary
Paper summary: Merlin:Empowering Multimodal LLMs with Foresight Minds
Merlin trains a multimodal LLM to predict object trajectories and then reason about future events, using two stages called Foresight Pre-Training and Foresight Instruction-Tuning.
- Trajectories of objects across frames give a structured target for teaching an MLLM "what happens next".
- The model first predicts trajectories, then reasons about future events from them.