#generation

• 21 Jan 2026 · Study

  LLMs · Decoding: greedy, beam, sampling

  How a model's next-token probabilities become text: greedy, beam search, temperature, top-k and top-p sampling, compared on GPT-2.

• 25 Nov 2025 · Study

  Transformers · GPT

  GPT as a decoder-only Transformer trained to predict the next token; generating with GPT-2, measuring perplexity, and seeing why causal masking makes this possible.