Gradient Descent Optimization & Challenges

YouTube search... ...Google search

• Gradient Boosting Algorithms
• Backpropagation
• Objective vs. Cost vs. Loss vs. Error Function
• Topology and Weight Evolving Artificial Neural Network (TWEANN)

Gradient Descent - Stochastic (SGD), Batch (BGD) & Mini-Batch

Youtube search...

Vanishing & Exploding Gradients Problems

YouTube Search

• Vanishing gradient problem

Vanishing & Exploding Gradients Challenges with Long Short-Term Memory (LSTM) and Recurrent Neural Networks (RNN)

[YouTube Search]

• Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Recurrent Neural Network (RNN)