#small-models
- 1 Jan 2026 · Paper summary
Paper summary: Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models
Youtu-LLM is a 1.96B model pre-trained from scratch (not distilled) with a commonsense → STEM → agent curriculum over about 11T tokens and agentic mid-training, reported as state of the art below 2B parameters.
- 8 Oct 2025 · Paper summary
Paper summary: Less is More: Recursive Reasoning with Tiny Networks
Tiny Recursive Model, a single 2-layer network with 7M parameters that repeatedly refines its answer, reports 45% on ARC-AGI-1 and 8% on ARC-AGI-2, above many large LLMs on these puzzles.
- 8 Jul 2025 · AI updates
SmolLM3: an open 3B reasoner with the full recipe
Hugging Face released SmolLM3, a 3B model trained on 11T tokens with dual think/no_think modes, six languages and 128K context, published with its complete training blueprint.
- 27 May 2025 · Paper summary
Paper summary: Mutarjim: Advancing Bidirectional Arabic-English Translation with a Small Language Model
Mutarjim is a 1.5B-parameter Arabic–English translation model that's reported to rival models up to 20× larger, plus Tarjama-25, a new 5,000-pair benchmark built to fix gaps in existing test sets.
- 8 Apr 2025 · Paper summary
Paper summary: SmolVLM: Redefining small and efficient multimodal models
SmolVLM is a family of compact vision-language models designed for on-device inference; the 256M model uses under 1GB of GPU memory and is reported to beat the 300× larger Idefics-80B.
- 17 Mar 2025 · Paper summary
Paper summary: SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
SmolDocling is a 256M-parameter vision-language model that converts whole document pages end to end into DocTags, a markup capturing content, structure and location of every element.
- 6 Feb 2025 · Paper summary
Paper summary: SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
SmolLM2 is a 1.7B model "overtrained" on about 11 trillion tokens in stages that mix web, maths, code and instruction data, with new datasets (FineMath, Stack-Edu, SmolTalk) built where existing ones were too small or noisy.
- 9 Jan 2025 · Paper summary
Paper summary: rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking
rStar-Math gets small language models to strong maths reasoning with Monte Carlo Tree Search guided by a small process preference model, both improved through self-evolution without distillation from stronger models.
- 27 Dec 2024 · Paper summary
Paper summary: YuLan-Mini: An Open Data-efficient Language Model
YuLan-Mini is a 2.42B-parameter base model trained on 1.08T tokens that the authors report matches models trained on much more data, with full details of the data and training released.
- 23 Apr 2024 · Paper summary
Paper summary: Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
phi-3-mini is a 3.8B model trained on 3.3T tokens of heavily filtered web and synthetic data, reported to reach 69% on MMLU while being small enough to run on a phone.
- 12 Sept 2023 · Paper summary
Paper summary: Textbooks Are All You Need II: phi-1.5 technical report
phi-1.5 is a 1.3B-parameter model trained mostly on synthetic "textbook-quality" data, reported to match models about five times larger on common-sense reasoning.