Paper summary: Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models

Problem

Small models usually get their abilities by distillation from larger ones. Can a small model learn reasoning and planning natively in pre-training?

Method

Youtu-LLM (1.96B) is pre-trained from scratch with:

1. a compact dense Multi-Latent Attention architecture with a STEM-oriented vocabulary and a 128k context window
2. a “Commonsense–STEM–Agent” curriculum over about 11T tokens, shifting the data from general commonsense to complex STEM and then agentic tasks
3. agentic mid-training on synthesised trajectories across maths, coding and tool use, to build planning and reflection behaviours

Key results

From the abstract: a new state of the art for sub-2B LLMs, competitive with larger models on general benchmarks, and significantly above existing baselines on agent-specific tasks.

Limitations

There’s no dedicated limitations section in the extracted text. Comparisons are the developers’ own.