Paper summary
Paper summary: 2.5 Years in Class: A Multimodal Textbook for Vision-Language Pretraining
The authors turn 22,000 class hours of instructional videos into an interleaved image–text "textbook" corpus for vision-language pre-training, with denser knowledge than web-crawled interleaved data.
- Web-crawled image–text data often has loosely related images and text; lecture videos keep them tightly linked.
- Keyframes, speech transcripts (ASR) and OCR text are extracted and refined into interleaved samples.