Linear Least Squares

Least Squares (LS) is the common name to denote a mathematical procedure for finding the best-fitting mathematical model to a given set of observations by minimizing the sum of the model's squared residuals, a residual being the difference between an observed value and the fitted value provided by a model.

Least Squares problems fall into two categories, depending on whether or not the residuals are linear in all unknowns:

• Linear LS in several forms which commonly occurs in statistical regression analysis and has a closed-form analytical solution.

• Nonlinear LS which usually has no analytical solution. Instead, iterative numerical algorithms are used to find the optimum.

This section is dedicated to Linear LS problem.

Implementation of the LS principle is considered in different forms starting from the simplest canonical model towards to more complicated models.

Please refer to the Nonlinear Least Squares (NLS) Problem chapter for details about Nonlinear LS solver.