(Deep) Residual Network (DRN) - ResNet

YouTube search... ...Google search

• ResNet-50
• Deep Learning
• Recurrent Neural Network (RNN)
• Google DeepMind AlphaGo Zero
• Protein Folding & Discovery ...Google DeepMind AlphaFold
• Feed Forward Neural Network (FF or FFNN)
• Deep Residual Learning for Image Recognition | K. He, X. Zhang, S. Ren, and J. Sun
• Neural Network Zoo | Fjodor Van Veen

Deep residual networks (DRN); called ResNets, are very deep Feed Forward Neural Networks (FFNNs) with extra connections; callled 'skip connections', passing input from one layer to a later layer (often 2 to 5 layers) as well as the next layer. Instead of trying to find a solution for mapping some input to some output across say 5 layers, the network is enforced to learn to map some input to some output + some input. Basically, it adds an identity to the solution, carrying the older input over and serving it freshly to a later layer. It has been shown that these networks are very effective at learning patterns up to 150 layers deep, much more than the regular 2 to 5 layers one could expect to train. However, it has been proven that these networks are in essence just Recurrent Neural Network (RNNs) without the explicit time based construction and they’re often compared to Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Recurrent Neural Network (RNN) without gates. Deep Residual Learning for Image Recognition | Kaiming He Xiangyu Zhang Shaoqing Ren Jian Sun @ Microsoft Research