Transformer

YouTube search... ...Google search

• Sequence to Sequence (Seq2Seq)
• Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)
• Autoencoders / Encoder-Decoders
• Natural Language Processing (NLP)
• Memory Networks

Attention mechanisms in neural networks are about memory access. That’s the first thing to remember about attention: it’s something of a misnomer. A Beginner's Guide to Attention Mechanisms and Memory Networks | Skymind