Hamming code

In his original paper, Hamming elaborated his general idea, but specifically focused on the Hamming(7,4) code which adds three parity bits to four bits of data.

The parity-check matrix has the property that any two columns are pairwise linearly independent.

In this context, an extended Hamming code having one extra parity bit is often used.

Extended Hamming codes achieve a Hamming distance of four, which allows the decoder to distinguish between when at most one one-bit error occurs and when any two-bit errors occur.

In this sense, extended Hamming codes are single-error correcting and double-error detecting, abbreviated as SECDED.

Richard Hamming, the inventor of Hamming codes, worked at Bell Labs in the late 1940s on the Bell Model V computer, an electromechanical relay-based machine with cycle times in seconds.

Input was fed in on punched paper tape, seven-eighths of an inch wide, which had up to six holes per row.

During weekdays, when errors in the relays were detected, the machine would stop and flash lights so that the operators could correct the problem.

During after-hours periods and on weekends, when there were no operators, the machine simply moved on to the next job.

Hamming worked on weekends, and grew increasingly frustrated with having to restart his programs from scratch due to detected errors.

[3] Over the next few years, he worked on the problem of error-correction, developing an increasingly powerful array of algorithms.

In 1950, he published what is now known as Hamming code, which remains in use today in applications such as ECC memory.

Parity adds a single bit that indicates whether the number of ones (bit-positions with values of one) in the preceding data was even or odd.

However, while the quality of parity checking is poor, since it uses only a single bit, this method results in the least overhead.

Moreover, increasing the size of the parity bit string is inefficient, reducing throughput by three times in our original case, and the efficiency drops drastically as we increase the number of times each bit is duplicated in order to detect and correct more errors.

Hamming studied the existing coding schemes, including two-of-five, and generalized their concepts.

The (3,1) repetition has a distance of 3, as three bits need to be flipped in the same triple to obtain another code word with no visible errors.

When three bits flip in the same group there can be situations where attempting to correct will produce the wrong code word.

During the 1940s he developed several encoding schemes that were dramatic improvements on existing codes.

The key to all of his systems was to have the parity bits overlap, such that they managed to check each other as well as the data.

The following general algorithm generates a single-error correcting (SEC) code for any number of bits.

To remedy this shortcoming, Hamming codes can be extended by an extra parity bit.

This extended Hamming code was popular in computer memory systems, starting with IBM 7030 Stretch in 1961,[4] where it is known as SECDED (or SEC-DED, abbreviated from single error correction, double error detection).

[5] Server computers in 21st century, while typically keeping the SECDED level of protection, no longer use Hamming's method, relying instead on the designs with longer codewords (128 to 256 bits of data) and modified balanced parity-check trees.

[4] The (72,64) Hamming code is still popular in some hardware designs, including Xilinx FPGA families.

With the addition of an overall parity bit, it becomes the [8,4] extended Hamming code and can both detect and correct single-bit errors and detect (but not correct) double-bit errors.

The parity-check matrix H of a Hamming code is constructed by listing all columns of length m that are pair-wise independent.

So G can be obtained from H by taking the transpose of the left hand side of H with the identity k-identity matrix on the left hand side of G. The code generator matrix

Finally, these matrices can be mutated into equivalent non-systematic codes by the following operations:[6] From the above matrix we have 2k = 24 = 16 codewords.

Using the systematic construction for Hamming codes from above, the matrix A is apparent and the systematic form of G is written as The non-systematic form of G can be row reduced (using elementary row operations) to match this matrix.

Graphical depiction of the four data bits and three parity bits and which parity bits apply to which data bits
The same [7,4] example from above with an extra parity bit. This diagram is not meant to correspond to the matrix H for this example.