Paper summary: Vidu S1: A Real-Time Interactive Video Generation Model

Vidu S1 generates infinite-length interactive video of digital characters in real time, controlled by voice at any moment, at 540p and up to 42 FPS on consumer GPUs.

Figure from the paper "Vidu S1: A Real-Time Interactive Video Generation Model"

Figure from Zhang et al. (2026), licensed CC BY 4.0. Figure 1 : Overview of Vidu S1 . Vidu S1 supports speech-guided future control, efficient infinite-duration inference, and diverse customized characters for real-time interactive video generation.

Problem

Most video generation is offline: write a prompt, wait for a clip. Interactive characters need continuous, real-time generation that responds to control instantly, without drifting over time.

Method

Vidu S1 (Figure 1):

voice instructions can change the video content at any moment
infinite-length generation, designed to avoid blurring, drift or distortion
built with TurboDiffusion and TurboServe for speed
users can upload custom images (real people, anime, pets) and choose voice tones

Key results

From the abstract: 540p real-time video at up to 42 FPS on regular consumer GPUs, and the best performance across all test metrics in the authors’ experiments. A public demo is available.

Limitations

There’s no dedicated limitations section in the extracted text. Generating real people’s likeness from uploaded photos raises obvious consent and misuse questions, which the abstract doesn’t discuss.