Study
NLP · Seq2seq
The encoder–decoder architecture for sequence-to-sequence tasks, teacher forcing, and a small GRU model learning to reverse digit strings.
- An encoder compresses the input into a vector; a decoder generates the output one token at a time from it.
- The fixed-size bottleneck vector is the weakness that attention was invented to fix.