Difference between revisions of "Bidirectional Long Short-Term Memory (BI-LSTM)"

From
Jump to: navigation, search
Line 17: Line 17:
  
 
The purpose of the Bi-LSTM is to look at a particular sequences both from front-to-back as well as from back-to-front. In this way, the network creates a context for each character in the text that depends on both its past as well as its future.
 
The purpose of the Bi-LSTM is to look at a particular sequences both from front-to-back as well as from back-to-front. In this way, the network creates a context for each character in the text that depends on both its past as well as its future.
 +
 +
Bidirectional Long/Short-Term Memory (BiLSTM) look exactly the same as its unidirectional counterpart. The difference is that the network is not just connected to the past, but also to the future. As an example, unidirectional LSTMs might be trained to predict the word “fish” by being fed the letters one by one, where the recurrent connections through time remember the last value. A BiLSTM would also be fed the next letter in the sequence on the backward pass, giving it access to future information. This trains the network to fill in gaps instead of advancing information, so instead of expanding an image on the edge, it could fill a hole in the middle of an image.  Schuster, Mike, and Kuldip K. Paliwal. “Bidirectional recurrent neural networks.” IEEE Transactions on Signal Processing 45.11 (1997): 2673-2681.
 +
  
 
<youtube>fKSijaXq9qM</youtube>
 
<youtube>fKSijaXq9qM</youtube>
 
<youtube>Bp-_DatyUCY</youtube>
 
<youtube>Bp-_DatyUCY</youtube>

Revision as of 12:27, 11 June 2020

YouTube search... ...Google search

The purpose of the Bi-LSTM is to look at a particular sequences both from front-to-back as well as from back-to-front. In this way, the network creates a context for each character in the text that depends on both its past as well as its future.

Bidirectional Long/Short-Term Memory (BiLSTM) look exactly the same as its unidirectional counterpart. The difference is that the network is not just connected to the past, but also to the future. As an example, unidirectional LSTMs might be trained to predict the word “fish” by being fed the letters one by one, where the recurrent connections through time remember the last value. A BiLSTM would also be fed the next letter in the sequence on the backward pass, giving it access to future information. This trains the network to fill in gaps instead of advancing information, so instead of expanding an image on the edge, it could fill a hole in the middle of an image. Schuster, Mike, and Kuldip K. Paliwal. “Bidirectional recurrent neural networks.” IEEE Transactions on Signal Processing 45.11 (1997): 2673-2681.