#positional-encoding
- 28 Oct 2025 · Study
Transformers · Positional encoding
Why attention needs position information, the sinusoidal encoding from the original paper, learned position embeddings, and the relative-position idea behind RoPE.
Why attention needs position information, the sinusoidal encoding from the original paper, learned position embeddings, and the relative-position idea behind RoPE.