Sahaj.blogs

Day page

11 Feb 2026

Paper summary

Paper summary: OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration

OPUS picks pre-training data at every iteration by projecting each candidate's optimizer-shaped update onto a target direction, at 4.7% extra compute; GPT-2 runs on 30B selected tokens beat full 200B-token training.

  • With high-quality text running out (the "data wall"), the goal shifts from more tokens to better tokens.
  • Utility is measured in the optimizer's update space (e.g. Adam-shaped), not on raw gradients.

Read the post →

Study

LLMs · RLHF

Reinforcement learning from human feedback in three stages, with the reward-model step built from scratch: a Bradley–Terry model learning to prefer the better of two answers from pairwise labels.

  • RLHF = SFT, then a reward model trained on human preference pairs, then RL (PPO) against that reward with a KL penalty.
  • The reward model only learns from comparisons: answer A is better than answer B.

Read the post →