AI updates
Mistral Large 4 ("Le Chonk"): ~1T-parameter MoE with sparse attention
Mistral Large 4 is a multimodal MoE of about 1T total parameters with a 1M-token context, combining expert sparsity with sparse attention and a Pixtral vision encoder, released in FP8 and NVFP4 under an MIT-derived open-weight licence.
- Two kinds of sparsity: a router picks experts for each token, and an attention indexer picks which earlier positions to attend to.
- The licence is MIT-derived but restricts model-as-a-service offerings.