Difference between revisions of "(Deep) Residual Network (DRN) - ResNet"
m (BPeat moved page Residual Neural Networks (ResNet) to Residual Neural Network (ResNet) without leaving a redirect) |
|||
| Line 1: | Line 1: | ||
[[http://www.youtube.com/results?search_query=residual+neural+networks+ResNet YouTube search...]] | [[http://www.youtube.com/results?search_query=residual+neural+networks+ResNet YouTube search...]] | ||
| + | |||
| + | * [http://www.asimovinstitute.org/author/fjodorvanveen/ Neural Network Zoo | Fjodor Van Veen] | ||
| + | |||
| + | Deep residual networks (DRN) are very deep FFNNs with extra connections passing input from one layer to a later layer (often 2 to 5 layers) as well as the next layer. Instead of trying to find a solution for mapping some input to some output across say 5 layers, the network is enforced to learn to map some input to some output + some input. Basically, it adds an identity to the solution, carrying the older input over and serving it freshly to a later layer. It has been shown that these networks are very effective at learning patterns up to 150 layers deep, much more than the regular 2 to 5 layers one could expect to train. However, it has been proven that these networks are in essence just RNNs without the explicit time based construction and they’re often compared to LSTMs without gates. He, Kaiming, et al. “Deep residual learning for image recognition.” arXiv preprint arXiv:1512.03385 (2015). | ||
<youtube>GSsKdtoatm8</youtube> | <youtube>GSsKdtoatm8</youtube> | ||
<youtube>9w13O3gwUOQ</youtube> | <youtube>9w13O3gwUOQ</youtube> | ||
<youtube>hwMsKmgopSU</youtube> | <youtube>hwMsKmgopSU</youtube> | ||
| + | <youtube>UlnYEWXoxOY</youtube> | ||
Revision as of 16:35, 11 May 2018
Deep residual networks (DRN) are very deep FFNNs with extra connections passing input from one layer to a later layer (often 2 to 5 layers) as well as the next layer. Instead of trying to find a solution for mapping some input to some output across say 5 layers, the network is enforced to learn to map some input to some output + some input. Basically, it adds an identity to the solution, carrying the older input over and serving it freshly to a later layer. It has been shown that these networks are very effective at learning patterns up to 150 layers deep, much more than the regular 2 to 5 layers one could expect to train. However, it has been proven that these networks are in essence just RNNs without the explicit time based construction and they’re often compared to LSTMs without gates. He, Kaiming, et al. “Deep residual learning for image recognition.” arXiv preprint arXiv:1512.03385 (2015).
