In the formulation of practical optimization methods,it is often the case that the choice of numericallinear algebra method used in some inherent calculationcan determine the choice of the whole optimizationalgorithm. The numerical linear algebra is particularlyrelevant in large-scale optimization, where the linearequation solver has a dramatic effect on both therobustness and the efficiency of the optimization.
We review some of the principal linear algebraic issuesassociated with the design of modern optimizationalgorithms. Much of the discussion will concern the useof direct and iterative linear solvers for large-scaleoptimization. Particular emphasis will be given to somerecent developments in the use of regularization.