Numerical solution to multidimensional non-linear optimization

Basic idea, start at a point P in N-dimensional space, and we proceed from there in some vector direction n, then any function of N variables f(P) can be minimized along the line n by our one-dimensional methods. So a multidimensional minimization methods can be transformed as sequences of line minimizations. Different methods differ only by how, at each stage, they choose the next direction n to try.

The basic routine of line minimization is

linmin: Given as input the vectors P and n, and the function f, find the scalar /lambda that minimizes f(P+\lambda*n). Replace P by P+\lambda*n, replace n by \lambda*n. Done.