Conjugate Gradient Methods in Multidimensional non-linear Optimization

Use gradient information:

f(x)~=c-b*x+1/2x*A*x.

Steepest Descent algorithm....not so good.

Solution: Conjugate Gradient Method.

Minimizing the function

f(x)=1/2 *x*A*x-b*x.

The function is minimized when its gradient

gradient(f)=A*x-b is zero.

The conjugate gradient method can be used to solve not only linear algebraic equations by minimizing a quadratic form, but also minimizing a non-linear function.

*Conjugate directions
--"non-interfering" directions with special property that minimizing along one is not "spoiled" by subsequently minimizing along another.

In conjugate gradient method, we want to find new gradient in a direction that is "conjugate" to the old gradient, and to all previous directions traversed.