Paper summary: Qwen3-Omni Technical Report

Problem

Multimodal models often get worse at individual modalities as more are added. The goal here is one model across text, images, audio and video with no loss compared with single-modal models.

Method

This follows Qwen2.5-Omni (summarised here in March) with a Thinker–Talker MoE architecture:

• the Talker autoregressively predicts discrete speech codec tokens with a multi-codebook scheme
• a lightweight causal ConvNet replaces compute-heavy block-wise diffusion for decoding speech, so streaming can start from the first codec frame
• a Thinking model reasons explicitly over inputs from any modality
• a fine-tuned Captioner gives detailed audio captions, because the authors say the community lacked a general-purpose one

Key results

From the abstract:

• matches same-sized single-modal Qwen models, and is especially strong on audio
• across 36 audio and audio-visual benchmarks: open-source state of the art on 32, overall state of the art on 22, beating Gemini-2.5-Pro, Seed-ASR and GPT-4o-Transcribe
• 119 text languages, 19 for speech understanding, 10 for speech generation
• theoretical first-packet latency of 234 ms
• the 30B-A3B models are released under Apache 2.0

Limitations

There’s no dedicated limitations section in the extracted text. The latency figure is theoretical, under cold-start conditions.