LU decomposition

Sometimes factorization is impossible without prior reordering of A to prevent division by zero or uncontrolled growth of rounding errors hence alternative expression becomes:

Then the system of equations has the following solution: Substituting these values into the LU decomposition above yields Any square matrix

is invertible, then it admits an LU (or LDU) factorization if and only if all its leading principal minors[6] are nonzero[7] (for example

For a (not necessarily invertible) matrix over any field, the exact necessary and sufficient conditions under which it has an LU factorization are known.

The Gaussian elimination algorithm for obtaining LU decomposition has also been extended to this most general case.

[10] When an LDU factorization exists and is unique, there is a closed (explicit) formula for the elements of L, D, and U in terms of ratios of determinants of certain submatrices of the original matrix A.

, we may start by swapping rows to provide the desired conditions for the n-th column.

Because we are implementing partial pivoting, we swap the second and third rows of our derived matrix and the current version of our

Although Banachiewicz (1938) LU decomposition algorithm preceded advent of programmed electronic computers, yet it was ready made for direct implementation into code as index swapping, transpose and column by column multiplication remain native built capabilities of the most programming languages and are handled by compilers alone with little delay of actual execution.

The peculiar matrix notation used by Banachiewicz enabled him to multiply matrices column by column, a convenient feature for mechanical calculations as he could reveal consecutive factors by sliding a ruler to next rows of matrices.

Next calculations continue for the subsequent rows and columns till the bottom right corner of A.

The figure illustrates calculations for 3-rd row and column, assuming previous stages were already completed.

Green filled thin frame boxes indicate values already known, from previous stages.

This enables replacement of these elements with the result values of U and L, i.e. execution of LU decomposition in place, so that the whole A is replaced with U and L except for the unit diagonal of L. Banachiewicz LU algorithm is well suited for partial pivoting by choosing the absolute maximum pivot from the newly calculated row of U and subsequently swapping its columns so that it lands on the main diagonal.

All partial pivoting LU algorithms cost roughly the same amount, of order

Ideally, the cost of computation is determined by the number of nonzero entries, rather than by the size of the matrix.

General treatment of orderings that minimize fill-in can be addressed using graph theory.

In this case the solution is done in two logical steps: In both cases we are dealing with triangular matrices (L and U), which can be solved directly by forward and backward substitution without using the Gaussian elimination process (however we do need this process or equivalent to compute the LU decomposition itself).

In this case it is faster (and more convenient) to do an LU decomposition of the matrix A once and then solve the triangular matrices for the different b, rather than using Gaussian elimination each time.

[16] When solving systems of equations, b is usually treated as a vector with a length equal to the height of matrix A.

also equals the right-hand side of the above equation, if we let S be the total number of row and column exchanges.

The same method readily applies to LU decomposition by setting P equal to the identity matrix.

The LU decomposition is related to elimination of linear systems of equations, as e.g. described by Ralston.

[19] Before Gauss many mathematicians in Eurasia were performing and perfecting it yet as the method became relegated to school grade, few of them left any detailed descriptions.

[4] To quote: "It appears that Gauss and Doolittle applied the method [of elimination] only to symmetric equations.

More recent authors, for example, Aitken, Banachiewicz, Dwyer, and Crout … have emphasized the use of the method, or variations of it, in connection with non-symmetric problems … Banachiewicz … saw the point … that the basic problem is really one of matrix factorization, or “decomposition” as he called it.

"[20] Banachiewicz[4] was the first to consider elimination in terms of matrices and in this way formulated LU decomposition, as demonstrated by his graphic illustration.

Matrix formulae to calculate rows and columns of LU factors by recursion are given in the remaining part of Banachiewicz's paper as Eq.

They are sometimes confused as later publications tend to tie his name solely with the rediscovery of Cholesky decomposition.

Banachiewicz himself can be excused of inaction as already next year he suffered from persecution by occupiers, spending three month in the Sachsenhausen Concentration Camp, on release from which he carried himself from a train his collaborator and co-prisoner Antoni Wilk, who died of exhaustion a week later.