Unlike most other computational procedures, it is hard to tell if a particular decision-making method has derived the correct answer or not.
If the solutions exhibit some logic contradictions (in the form of undesirable rank reversals of the alternatives), then one may argue that something is wrong with the method that derived them.
The first type of rank reversal in the above context was observed by Belton and Gear in 1983 as part of a study [2] of the analytic hierarchy process (AHP).
A few years later it was observed that the AHP, as well as a new variant to it that was introduced by Professor Thomas Saaty (the inventor of the AHP) in response to the previous observation by Belton and Gear, may exhibit rank reversals when a non-optimal alternative is replaced by a worse one (and not a copy of an alternative as in Belton and Gear's experiment).
[4] The issue of rank reversals has captured the interest of many researchers and practitioners in the field of decision-making.
[5][6][7][8][9][10] There are many different types of rank reversals, depending on how the alternatives in a problem are defined and evaluated.
[1][11][12] The weighted product model (WPM) does not exhibit the previous types of rank reversals, due to the multiplication formula it uses.
[1][11] However, the WPM does cause rank reversals when it is compared with the weighted sum model (WSM) and under the condition that all the criteria of a given decision problem can be measured in exactly the same unit.
Many authors use them as means to criticize decision making methods or to better explain rational behavior.
The issue of having rank reversals by rational decision makers has been studied extensively by Amos Tversky.
[13] In other words, having rank reversals in certain occasions and of certain types may not be indicative to faulty decision making.
However, the key question is how to be able to distinguish when rank reversals indicate that something is wrong or when they do not conflict rational decision making.
The following is just a partial list of multi-criteria decision making methods which have been confirmed to exhibit various types of rank reversals:[1] [4][5][6][7][8][9][10][14][15][16][17]