Sahaj.blogs

Day page

1 Jan 2026

Paper summary

Paper summary: Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Models

Youtu-LLM is a 1.96B model pre-trained from scratch (not distilled) with a commonsense → STEM → agent curriculum over about 11T tokens and agentic mid-training, reported as state of the art below 2B parameters.

  • The data distribution shifts during pre-training from general commonsense to STEM to agentic tasks.
  • A dedicated agentic mid-training stage uses synthesised maths, coding and tool-use trajectories.

Read the post →

Study

LLMs · Pre-training

Pre-training as next-token prediction on lots of text, shown end to end with a tiny character-level Transformer trained on a few paragraphs, before and after.

  • Pre-training is the same next-token objective as GPT-2, at enormous scale.
  • Even a tiny model trained for a minute goes from random characters to word-like text.

Read the post →