Scoring algorithm

Scoring algorithm, also known as Fisher's scoring,[1] is a form of Newton's method used in statistics to solve maximum likelihood equations numerically, named after Ronald Fisher.

be random variables, independent and identically distributed with twice differentiable p.d.f.

f ( y ; θ )

, and we wish to calculate the maximum likelihood estimator (M.L.E.)

First, suppose we have a starting point for our algorithm

, and consider a Taylor expansion of the score function,

: where is the observed information matrix at

Now, setting

and rearranging gives us: We therefore use the algorithm and under certain regularity conditions, it can be shown that

In practice,

is usually replaced by

, the Fisher information, thus giving us the Fisher Scoring Algorithm: Under some regularity conditions, if

is a consistent estimator, then

(the correction after a single step) is 'optimal' in the sense that its error distribution is asymptotically identical to that of the true max-likelihood estimate.