Paper summary: Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency

Problem

Diffusion models made audio-driven talking-portrait videos much more natural. But audio is only a weak control signal for body and head motion, so many methods add spatial signals (motion templates, constraints) to stabilise the movement. According to the authors, this can compromise how natural and free the motion is.

Method

Loopy is conditioned on audio only. Two components help it learn natural motion from data:

• an inter- and intra-clip temporal module, which lets the model use long-term motion information across clips
• an audio-to-latents module, which improves the correlation between audio and portrait movement

This removes the need for manually specified spatial motion templates at inference.

Key results

The abstract reports that Loopy outperforms recent audio-driven portrait diffusion methods, with more lifelike, high-quality results across scenarios.

Limitations

There’s no dedicated limitations section in the extracted text. Results are shown on portrait videos, and realistic avatar generation raises obvious misuse concerns (deepfakes).