Difference between revisions of "Bidirectional Long Short-Term Memory (BI-LSTM)"
m |
|||
| (7 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
|description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools | |description=Helpful resources for your journey with artificial intelligence; videos, articles, techniques, courses, profiles, and tools | ||
}} | }} | ||
| − | [ | + | [https://www.youtube.com/results?search_query=Bidirectional+LSTM YouTube search...] |
| − | [ | + | [https://www.google.com/search?q=Bidirectional+LSTM+machine+learning+ML+artificial+intelligence ...Google search] |
| + | * [[Memory]] | ||
* [[Recurrent Neural Network (RNN)]] Variants: | * [[Recurrent Neural Network (RNN)]] Variants: | ||
** [[Long Short-Term Memory (LSTM)]] | ** [[Long Short-Term Memory (LSTM)]] | ||
| + | ** [[Manhattan LSTM (MaLSTM)]] — a Siamese architecture based on recurrent neural network | ||
** [[Gated Recurrent Unit (GRU)]] | ** [[Gated Recurrent Unit (GRU)]] | ||
** Bidirectional Long Short-Term Memory (BI-LSTM) | ** Bidirectional Long Short-Term Memory (BI-LSTM) | ||
| Line 15: | Line 17: | ||
** [[Average-Stochastic Gradient Descent (SGD) Weight-Dropped LSTM (AWD-LSTM)]] | ** [[Average-Stochastic Gradient Descent (SGD) Weight-Dropped LSTM (AWD-LSTM)]] | ||
** [[Hopfield Network (HN)]] | ** [[Hopfield Network (HN)]] | ||
| + | * [[Attention]] Mechanism ...[[Transformer]] ...[[Generative Pre-trained Transformer (GPT)]] ... [[Generative Adversarial Network (GAN)|GAN]] ... [[Bidirectional Encoder Representations from Transformers (BERT)|BERT]] | ||
| + | |||
| + | The purpose of the Bi-LSTM is to look at a particular sequences both from front-to-back as well as from back-to-front. In this way, the network creates a [[context]] for each character in the text that depends on both its past as well as its future. | ||
| + | |||
| + | Bidirectional Long/Short-Term [[Memory]] (BiLSTM) look exactly the same as its unidirectional counterpart. The difference is that the network is not just connected to the past, but also to the future. As an example, unidirectional LSTMs might be trained to predict the word “fish” by being fed the letters one by one, where the recurrent connections through time remember the last value. A BiLSTM would also be fed the next letter in the sequence on the backward pass, giving it access to future information. This trains the network to fill in gaps instead of advancing information, so instead of expanding an image on the edge, it could fill a hole in the middle of an image. Schuster, Mike, and Kuldip K. Paliwal. “Bidirectional recurrent neural networks.” IEEE Transactions on Signal Processing 45.11 (1997): 2673-2681. | ||
| − | |||
<youtube>fKSijaXq9qM</youtube> | <youtube>fKSijaXq9qM</youtube> | ||
<youtube>Bp-_DatyUCY</youtube> | <youtube>Bp-_DatyUCY</youtube> | ||
Latest revision as of 22:16, 1 March 2024
YouTube search... ...Google search
- Memory
- Recurrent Neural Network (RNN) Variants:
- Long Short-Term Memory (LSTM)
- Manhattan LSTM (MaLSTM) — a Siamese architecture based on recurrent neural network
- Gated Recurrent Unit (GRU)
- Bidirectional Long Short-Term Memory (BI-LSTM)
- Bidirectional Long Short-Term Memory (BI-LSTM) with Attention Mechanism
- Average-Stochastic Gradient Descent (SGD) Weight-Dropped LSTM (AWD-LSTM)
- Hopfield Network (HN)
- Attention Mechanism ...Transformer ...Generative Pre-trained Transformer (GPT) ... GAN ... BERT
The purpose of the Bi-LSTM is to look at a particular sequences both from front-to-back as well as from back-to-front. In this way, the network creates a context for each character in the text that depends on both its past as well as its future.
Bidirectional Long/Short-Term Memory (BiLSTM) look exactly the same as its unidirectional counterpart. The difference is that the network is not just connected to the past, but also to the future. As an example, unidirectional LSTMs might be trained to predict the word “fish” by being fed the letters one by one, where the recurrent connections through time remember the last value. A BiLSTM would also be fed the next letter in the sequence on the backward pass, giving it access to future information. This trains the network to fill in gaps instead of advancing information, so instead of expanding an image on the edge, it could fill a hole in the middle of an image. Schuster, Mike, and Kuldip K. Paliwal. “Bidirectional recurrent neural networks.” IEEE Transactions on Signal Processing 45.11 (1997): 2673-2681.
