Study
LLMs · Decoding: greedy, beam, sampling
How a model's next-token probabilities become text: greedy, beam search, temperature, top-k and top-p sampling, compared on GPT-2.
- The model only gives probabilities; the decoding strategy decides what text you actually get.
- Temperature reshapes the distribution: below 1 sharpens it, above 1 flattens it.