#open-weights
- 29 Oct 2026 · AI updates
Mistral Large 4 ("Le Chonk"): ~1T-parameter MoE with sparse attention
Mistral Large 4 is a multimodal MoE of about 1T total parameters with a 1M-token context, combining expert sparsity with sparse attention and a Pixtral vision encoder, released in FP8 and NVFP4 under an MIT-derived open-weight licence.
- 28 Jul 2026 · Paper summary
Paper summary: Kimi K3: Open Frontier Intelligence
Kimi K3 is an open-weight 2.8T-parameter MoE (104B active) with native vision and a 1M-token context, reported as about 2.5× more scaling-efficient than Kimi K2, though still behind the strongest proprietary models.
- 15 Jul 2026 · AI updates
Inkling: a ~1T-parameter open multimodal model with 1M context
Thinking Machines released Inkling on Hugging Face, a ~1T-parameter decoder-only multimodal MoE (41B active) that takes image, text and audio, has a 1M context, and was trained on 45T tokens; a smaller Inkling-Small followed.
- 2 Apr 2026 · AI updates
Gemma 4: open multimodal models under Apache 2.0
Google DeepMind's Gemma 4 family is Apache 2.0, accepts images or video plus text (and audio for the small models), and adds Per-Layer Embeddings and a shared KV cache for efficiency.
- 5 Aug 2025 · AI updates
gpt-oss: OpenAI releases open-weight models under Apache 2.0
OpenAI released gpt-oss-120b and gpt-oss-20b, mixture-of-experts reasoning models quantised with MXFP4 and licensed Apache 2.0; the large one fits on one H100 and the small one in 16GB.
- 19 May 2025 · Paper summary
Paper summary: Qwen3 Technical Report
Qwen3 merges a "thinking" mode for multi-step reasoning and a fast non-thinking mode in one model family (0.6B to 235B, dense and MoE), adds a thinking budget, and expands to 119 languages.
- 12 Mar 2025 · AI updates
Gemma 3: multimodal, 128K context, 140+ languages
Google's Gemma 3 comes in 1B, 4B, 12B and 27B sizes; the 4B and up accept images, support 128K context and 140+ languages, and the 27B instruct model is reported to beat Gemini 1.5 Pro.
- 20 Dec 2024 · Paper summary
Paper summary: Qwen2.5 Technical Report
Qwen2.5 scales pre-training data from 7 to 18 trillion tokens and uses over a million SFT samples plus multistage RL, across a wide range of open-weight sizes.
- 23 Jul 2024 · AI updates
Llama 3.1: 405B open weights, long context, tool calling
Meta released Llama 3.1 in 8B, 70B and 405B sizes with multilingual support and long context, plus Llama Guard 3 and Prompt Guard; Hugging Face provided FP8, AWQ and GPTQ quantised versions.
- 16 Jul 2024 · Paper summary
Paper summary: Qwen2 Technical Report
The Qwen2 report covers open dense models from 0.5B to 72B plus an MoE, with Qwen2-72B reaching 84.2 on MMLU as a base model and support for about 30 languages.
- 27 Jun 2024 · AI updates
Gemma 2: 9B and 27B with sliding-window attention and distillation
Google released Gemma 2 in 9B and 27B sizes, trained on about twice the data of Gemma 1 and using sliding-window attention, logit soft-capping, knowledge distillation and model merging.
- 18 Apr 2024 · AI updates
Meta releases Llama 3 (8B and 70B)
Meta released Llama 3 in 8B and 70B sizes, base and instruct, plus Llama Guard 2 for safety classification, with an 8K context and day-one Hugging Face support.
- 21 Feb 2024 · AI updates
Google releases Gemma, open models based on Gemini
Google released Gemma, open LLMs in 2B and 7B sizes (base and instruction-tuned) with an 8K context, built from Gemini research, with day-one Hugging Face support.
- 11 Dec 2023 · AI updates
Mixtral 8x7B arrives on Hugging Face
Mistral released Mixtral 8x7B, a sparse mixture-of-experts LLM under Apache 2.0; Hugging Face integrated it into Transformers, TGI and Inference Endpoints on launch day.