Numerical differentiation
The simplest method is to use finite difference approximations.
A simple two-point estimation is to compute the slope of a nearby secant line through the points (x, f(x)) and (x + h, f(x + h)).
Therefore, the true derivative of f at x is the limit of the value of the difference quotient as the secant lines get closer and closer to being a tangent line:
indeterminate form, calculating the derivative directly can be unintuitive.
Another two-point formula is to compute the slope of a nearby secant line through the points (x − h, f(x − h)) and (x + h, f(x + h)).
In this case the first-order errors cancel, so the slope of these secant lines differ from the slope of the tangent line by an amount that is approximately proportional to
Hence for small values of h this is a more accurate approximation to the tangent line than the one-sided estimation.
[2][3] An important consideration in practice when the function is calculated using floating-point arithmetic of finite precision is the choice of step size, h. If chosen too small, the subtraction will yield a large rounding error.
In fact, all the finite-difference formulae are ill-conditioned[4] and due to cancellation will produce a value of zero if h is small enough.
[5] If too large, the calculation of the slope of the secant line will be more accurately calculated, but the estimate of the slope of the tangent by using the secant could be worse.
[6] For basic central differences, the optimal step is the cube-root of machine epsilon.
[7] For the numerical derivative formula evaluated at x and x + h, a choice for h that is small without producing a large rounding error is
(though not when x = 0), where the machine epsilon ε is typically of the order of 2.2×10−16 for double precision.
For computer calculations the problems are exacerbated because, although x necessarily holds a representable floating-point number in some precision (32 or 64-bit, etc.
This means that x + h will be changed (by rounding or truncation) to a nearby machine-representable number, with the consequence that (x + h) − x will not equal h; the two function evaluations will not be exactly h apart.
In this regard, since most decimal fractions are recurring sequences in binary (just as 1/3 is in decimal) a seemingly round step such as h = 0.1 will not be a round number in binary; it is 0.000110011001100...2 A possible approach is as follows: However, with computers, compiler optimization facilities may fail to attend to the details of actual computer arithmetic and instead apply the axioms of mathematics to deduce that dx and h are the same.
With C and similar languages, a directive that xph is a volatile variable will prevent this.
To obtain more general derivative approximation formulas for some function
which is called the three-point forward difference formula for the derivative.
By a similar approach, the five point midpoint approximation formula can be derived as:[10]
The classical finite-difference approximations for numerical differentiation are ill-conditioned.
is a holomorphic function, real-valued on the real line, which can be evaluated at points in the complex plane near
The recommended step size to obtain accurate derivatives for a range of conditions is
[18] In general, derivatives of any order can be calculated using Cauchy's integral formula:[19]
Using complex variables for numerical differentiation was started by Lyness and Moler in 1967.
A method based on numerical inversion of a complex Laplace transform was developed by Abate and Dubner.
[21] An algorithm that can be used without requiring knowledge about the method or the character of the function was developed by Fornberg.
[4] Differential quadrature is the approximation of derivatives by using weighted sums of function values.
[22][23] Differential quadrature is of practical interest because its allows one to compute derivatives from noisy data.
There are various methods for determining the weight coefficients, for example, the Savitzky–Golay filter.
