Difference between revisions of "Transformer"
| Line 2: | Line 2: | ||
[http://www.google.com/search?q=attention+model+deep+machine+learning+ML ...Google search] | [http://www.google.com/search?q=attention+model+deep+machine+learning+ML ...Google search] | ||
| − | *[[Sequence to Sequence (Seq2Seq)]] | + | * [[Sequence to Sequence (Seq2Seq)]] |
| − | *[[Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)]] | + | * [[Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)]] |
| − | *[[Autoencoders / Encoder-Decoders]] | + | * [[Autoencoders / Encoder-Decoders]] |
| − | *[[Natural Language Processing (NLP)]] | + | * [[Natural Language Processing (NLP)]] |
| + | * [http://skymind.ai/wiki/attention-mechanism-memory-network A Beginner's Guide to Attention Mechanisms and Memory Networks | Skymind] | ||
| + | |||
“Attend” to specific parts of the input (an image or text) in sequence, one after another. By relying on a sequence of glances, they capture (visual) structure, can be contrasted with other (machine vision) techniques that process a whole input e.g. image in a single, forward pass. | “Attend” to specific parts of the input (an image or text) in sequence, one after another. By relying on a sequence of glances, they capture (visual) structure, can be contrasted with other (machine vision) techniques that process a whole input e.g. image in a single, forward pass. | ||
| + | |||
| + | |||
| + | http://skymind.ai/images/wiki/attention_mechanism.png | ||
| + | http://skymind.ai/images/wiki/attention_model.png | ||
<youtube>W2rWgXJBZhU</youtube> | <youtube>W2rWgXJBZhU</youtube> | ||
| Line 13: | Line 19: | ||
<youtube>omHLeV1aicw</youtube> | <youtube>omHLeV1aicw</youtube> | ||
<youtube>IxQtK2SjWWM</youtube> | <youtube>IxQtK2SjWWM</youtube> | ||
| + | <youtube>XrZ_Y4koV5A</youtube> | ||
| + | <youtube>OYygPG4d9H0</youtube> | ||
Revision as of 10:42, 9 January 2019
YouTube search... ...Google search
- Sequence to Sequence (Seq2Seq)
- Recurrent Neural Networks (RNN) and Long Short-Term Memory (LSTM)
- Autoencoders / Encoder-Decoders
- Natural Language Processing (NLP)
- A Beginner's Guide to Attention Mechanisms and Memory Networks | Skymind
“Attend” to specific parts of the input (an image or text) in sequence, one after another. By relying on a sequence of glances, they capture (visual) structure, can be contrasted with other (machine vision) techniques that process a whole input e.g. image in a single, forward pass.
