Paper summary: Kimi K3: Open Frontier Intelligence

Problem

Pushing open-weight models to the frontier: very large scale, long context, native vision and strong agentic behaviour.

Method

Kimi K3 is a 2.8T-parameter Mixture-of-Experts model with 104B activated parameters, native vision and a 1M-token context window. Architectural pieces:

• Kimi Delta Attention and Attention Residuals, to improve information flow across sequence length and depth
• Stable LatentMoE, activating 16 of 896 routed experts per token

Post-training uses RL across general, agentic and coding domains at multiple reasoning-effort levels, including million-token agentic RL with persistent rollout and sandbox states. Infrastructure work covers kernel co-design, balanced expert-parallel training and deployment.

Key results

From the abstract:

• about 2.5× better overall scaling efficiency than Kimi K2
• frontier-level performance on long-horizon coding, agentic, knowledge, reasoning and vision tasks
• still trails the most powerful proprietary models, which the abstract names as Claude Fable 5 and GPT-5.6 Sol, while consistently outperforming the other open and proprietary models in the authors’ suite
• full weights released

Limitations

The abstract itself states the main one: it isn’t at the level of the strongest proprietary models. The licence on the paper text (CC BY-NC-ND) means its figures can’t be reused here.