Gradient descent is based on the observation that if the multi-variable function $${\displaystyle F(\mathbf {x} )}$$ is defined and differentiable in a neighborhood of a point $${\displaystyle \mathbf {a} }$$, … See more

Gradient descent can be used to solve a system of linear equations
$${\displaystyle A\mathbf {x} -\mathbf {b} =0}$$ See more

Gradient descent works in spaces of any number of dimensions, even in infinite-dimensional ones. In the latter case, the search space is typically a function space, and one calculates the Fréchet derivative of the functional to be minimized to determine the … See more

Gradient descent can be extended to handle constraints by including a projection onto the set of constraints. This method is only feasible when the projection is … See more

Gradient descent can also be used to solve a system of nonlinear equations. Below is an example that shows how to use the gradient descent to solve for three unknown variables, … See more

Gradient descent can converge to a local minimum and slow down in a neighborhood of a saddle point. Even for unconstrained quadratic minimization, gradient descent … See more

The properties of gradient descent depend on the properties of the objective function and the variant of gradient descent used (for example, if a line search step is used). The assumptions … See more