However, the exact threshold (level of decision) used to decide when a signal significantly emerges above the continuously fluctuating background noise remains arbitrary and is a matter of policy and often of debate among scientists, statisticians and regulators depending on the stakes in different fields.
[1][2][3] The detection limit is estimated from the mean of the blank, the standard deviation of the blank, the slope (analytical sensitivity) of the calibration plot and a defined confidence factor (e.g. 3.2 being the most accepted value for this arbitrary value).
[4] Another consideration that affects the detection limit is the adequacy and the accuracy of the model used to predict concentration from the raw analytical signal.
), "b" the value in which the straight line cuts the ordinates axis, "a" the sensitivity of the system (i.e., the slope of the line, or the function relating the measured signal to the quantity to be determined) and "x" the value of the quantity (e.g. temperature, concentration, pH, etc.)
Even when the same terminology is used, there can be differences in the LOD according to nuances of what definition is used and what type of noise contributes to the measurement and calibration.
For a signal at the LOD, the alpha error (probability of false positive) is small (1%).
However, the beta error (probability of a false negative) is 50% for a sample that has a concentration at the LOD (red line).
The instrument detection limit (IDL) is the analyte concentration that is required to produce a signal greater than three times the standard deviation of the noise level.
is the mean value of the signal for a reagent blank measured multiple times, and
The 3σ of the recorded absorbance signal can be considered as the detection limit for the specific element under the experimental conditions: selected wavelength, type of flame or graphite oven, chemical matrix, presence of interfering substances, instrument... .
For example, it might be necessary to heat a sample that is to be analyzed for a particular metal with the addition of acid first (digestion process).
The practical way for determining the MDL is to analyze seven samples of concentration near the expected limit of detection.
This estimation, however, ignores any uncertainty that arises from performing the sample preparation and will therefore probably underestimate the true MDL.
This explains the variety of definitions and the diversity of juridiction specific solutions developed to address preferences.
In many other disciplines such as geochemistry, seismology, astronomy, dendrochronology, climatology, life sciences in general, and in many other fields impossible to enumerate extensively, the problem is wider and deals with signal extraction out of a background of noise.
It involves complex statistical analysis procedures and therefore it also depends on the models used,[5] the hypotheses and the simplifications or approximations to be made to handle and manage uncertainties.
This explains why it is not easy to come to a general consensus, if any, about the precise mathematical definition of the expression of limit of detection.