vanishing gradient problem
The vanishing gradient problem refers to the issue in artificial neural networks where the gradient (a measure of how the network's parameters should be adjusted during training) becomes extremely small as it propagates backwards from the output layer to the earlier layers. This leads to very slow or no learning in those earlier layers, making it difficult for the network to effectively learn and update its parameters.
Requires login.
Related Concepts (1)
Similar Concepts
- accelerated gradient descent methods
- batch gradient descent
- conjugate gradient descent
- constraints in gradient descent optimization
- convergence of gradient descent
- gradient descent
- gradient descent for linear regression
- gradient descent for neural networks
- mini-batch gradient descent
- non-convex optimization using gradient descent
- online gradient descent
- optimization problems
- proximal gradient descent
- stochastic gradient descent
- variants of gradient descent algorithms