Constrained least squares

In constrained least squares one solves a linear least squares problem with an additional constraint on the solution.

[1][2] This means, the unconstrained equation

must be fit as closely as possible (in the least squares sense) while ensuring that some other property of

There are often special-purpose algorithms for solving such problems efficiently.

Some examples of constraints are given below: If the constraint only applies to some of the variables, the mixed problem may be solved using separable least squares[4] by letting

represent the unconstrained (1) and constrained (2) components.

Then substituting the least-squares solution for

, i.e. (where + indicates the Moore–Penrose pseudoinverse) back into the original expression gives (following some rearrangement) an equation that can be solved as a purely constrained problem in

is a projection matrix.

is obtained from the expression above.