Paper summary
Paper summary: Lumiere: A Space-Time Diffusion Model for Video Generation
Lumiere generates the full duration of a video in one pass with a Space-Time U-Net that downsamples in both space and time, instead of generating keyframes and then filling in between.
- Generating all frames at once, rather than keyframes plus temporal super-resolution, helps global motion stay coherent.
- The model reuses a pretrained text-to-image diffusion model and adds temporal down- and up-sampling.